2026-02-19T20:02:00.578 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-19T20:02:00.592 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-19T20:02:00.673 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606 branch: wip-yuri-testing-2026-02-18-1820 description: rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '59606' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial meta: - desc: run two ceph clusters and install rbd-mirror - desc: configure the permissions for client.mirror - desc: 2 ceph clusters with 1 mon and 3 osds each - desc: run four rbd-mirror daemons per cluster - desc: run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon name: yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri-testing-2026-02-18-1820 ceph: conf: client: debug rbd: 20 debug rbd_mirror: 15 log to stderr: false rbd default features: 125 rbd mirror image policy type: simple client.mirror.0: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.1: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.2: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.3: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2026-02-18-1820 sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - cluster1.mon.a - cluster1.mgr.x - cluster2.mgr.x - cluster1.osd.0 - cluster1.osd.1 - cluster1.osd.2 - cluster1.client.0 - cluster2.client.0 - - cluster2.mon.a - cluster2.osd.0 - cluster2.osd.1 - cluster2.osd.2 - cluster1.client.mirror - cluster1.client.mirror.0 - cluster1.client.mirror.1 - cluster1.client.mirror.2 - cluster1.client.mirror.3 - cluster1.client.mirror.4 - cluster1.client.mirror.5 - cluster1.client.mirror.6 - cluster2.client.mirror - cluster2.client.mirror.0 - cluster2.client.mirror.1 - cluster2.client.mirror.2 - cluster2.client.mirror.3 - cluster2.client.mirror.4 - cluster2.client.mirror.5 - cluster2.client.mirror.6 seed: 9494 sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-yuri-testing-2026-02-18-1820 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_802bf111cd55e57f5842153d7ba2b8b964aa818a/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a targets: trial073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB5jildMU4Gxv7Me0NWGejXuXKJU0ATaIZa5AK7cNxK30rrYQN2V8cPhtJ5LLP4cLUwEh81LB7fQXNqP3T3bdrk= trial080.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPW/JGWKJQToz3T5QPFuk5f7NULZNw/y2U6EqcR6aAdECz9epQf3o9nkHkkiTXJf2p8O4P+PHtQzkoVvKSwiJTI= tasks: - install: extra_packages: - rbd-mirror - ceph: cluster: cluster1 - ceph: cluster: cluster2 - exec: cluster1.client.mirror.0: - sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - rbd-mirror: client: cluster1.client.mirror.0 thrash: true - rbd-mirror: client: cluster1.client.mirror.1 thrash: true - rbd-mirror: client: cluster1.client.mirror.2 thrash: true - rbd-mirror: client: cluster1.client.mirror.3 thrash: true - rbd-mirror: client: cluster2.client.mirror.0 thrash: true - rbd-mirror: client: cluster2.client.mirror.1 thrash: true - rbd-mirror: client: cluster2.client.mirror.2 thrash: true - rbd-mirror: client: cluster2.client.mirror.3 thrash: true - rbd-mirror-thrash: cluster: cluster1 - rbd-mirror-thrash: cluster: cluster2 - workunit: clients: cluster1.client.mirror: - rbd/rbd_mirror_stress.sh env: CEPH_ARGS: '' MIRROR_POOL_MODE: image RBD_IMAGE_FEATURES: layering RBD_MIRROR_INSTANCES: '4' RBD_MIRROR_MODE: snapshot RBD_MIRROR_USE_EXISTING_CLUSTER: '1' RBD_MIRROR_USE_RBD_MIRROR: '1' timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-19_15:47:56 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-19T20:02:00.674 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_802bf111cd55e57f5842153d7ba2b8b964aa818a/qa; will attempt to use it 2026-02-19T20:02:00.674 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_802bf111cd55e57f5842153d7ba2b8b964aa818a/qa/tasks 2026-02-19T20:02:00.674 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-19T20:02:00.676 INFO:teuthology.task.internal:Checking packages... 2026-02-19T20:02:00.676 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '802bf111cd55e57f5842153d7ba2b8b964aa818a' 2026-02-19T20:02:00.676 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-19T20:02:00.676 INFO:teuthology.packaging:ref: None 2026-02-19T20:02:00.676 INFO:teuthology.packaging:tag: None 2026-02-19T20:02:00.676 INFO:teuthology.packaging:branch: wip-yuri-testing-2026-02-18-1820 2026-02-19T20:02:00.676 INFO:teuthology.packaging:sha1: 802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T20:02:00.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri-testing-2026-02-18-1820 2026-02-19T20:02:00.814 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5336.g802bf111 2026-02-19T20:02:00.816 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-19T20:02:00.827 INFO:teuthology.task.internal:no buildpackages task found 2026-02-19T20:02:00.827 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-19T20:02:00.840 INFO:teuthology.task.internal:Saving configuration 2026-02-19T20:02:00.856 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-19T20:02:00.868 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-19T20:02:00.944 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial073.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-19 19:58:33.482827', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB5jildMU4Gxv7Me0NWGejXuXKJU0ATaIZa5AK7cNxK30rrYQN2V8cPhtJ5LLP4cLUwEh81LB7fQXNqP3T3bdrk='} 2026-02-19T20:02:01.000 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial080.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-19 19:58:33.483385', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPW/JGWKJQToz3T5QPFuk5f7NULZNw/y2U6EqcR6aAdECz9epQf3o9nkHkkiTXJf2p8O4P+PHtQzkoVvKSwiJTI='} 2026-02-19T20:02:01.000 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-19T20:02:01.012 INFO:teuthology.task.internal:roles: ubuntu@trial073.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2026-02-19T20:02:01.012 INFO:teuthology.task.internal:roles: ubuntu@trial080.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2026-02-19T20:02:01.012 INFO:teuthology.run_tasks:Running task console_log... 2026-02-19T20:02:01.145 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x73ef6341e340>, signals=[15]) 2026-02-19T20:02:01.145 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-19T20:02:01.156 INFO:teuthology.task.internal:Opening connections... 2026-02-19T20:02:01.156 DEBUG:teuthology.task.internal:connecting to ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:02:01.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T20:02:01.217 DEBUG:teuthology.task.internal:connecting to ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:02:01.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T20:02:01.276 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-19T20:02:01.288 DEBUG:teuthology.orchestra.run.trial073:> uname -m 2026-02-19T20:02:01.303 INFO:teuthology.orchestra.run.trial073.stdout:x86_64 2026-02-19T20:02:01.304 DEBUG:teuthology.orchestra.run.trial073:> cat /etc/os-release 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:NAME="CentOS Stream" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:VERSION="9" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:ID="centos" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:ID_LIKE="rhel fedora" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:VERSION_ID="9" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:PLATFORM_ID="platform:el9" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:ANSI_COLOR="0;31" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:LOGO="fedora-logo-icon" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:HOME_URL="https://centos.org/" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-19T20:02:01.359 INFO:teuthology.orchestra.run.trial073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-19T20:02:01.360 INFO:teuthology.lock.ops:Updating trial073.front.sepia.ceph.com on lock server 2026-02-19T20:02:01.431 DEBUG:teuthology.orchestra.run.trial080:> uname -m 2026-02-19T20:02:01.447 INFO:teuthology.orchestra.run.trial080.stdout:x86_64 2026-02-19T20:02:01.447 DEBUG:teuthology.orchestra.run.trial080:> cat /etc/os-release 2026-02-19T20:02:01.503 INFO:teuthology.orchestra.run.trial080.stdout:NAME="CentOS Stream" 2026-02-19T20:02:01.503 INFO:teuthology.orchestra.run.trial080.stdout:VERSION="9" 2026-02-19T20:02:01.503 INFO:teuthology.orchestra.run.trial080.stdout:ID="centos" 2026-02-19T20:02:01.503 INFO:teuthology.orchestra.run.trial080.stdout:ID_LIKE="rhel fedora" 2026-02-19T20:02:01.503 INFO:teuthology.orchestra.run.trial080.stdout:VERSION_ID="9" 2026-02-19T20:02:01.503 INFO:teuthology.orchestra.run.trial080.stdout:PLATFORM_ID="platform:el9" 2026-02-19T20:02:01.504 INFO:teuthology.orchestra.run.trial080.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-19T20:02:01.504 INFO:teuthology.orchestra.run.trial080.stdout:ANSI_COLOR="0;31" 2026-02-19T20:02:01.504 INFO:teuthology.orchestra.run.trial080.stdout:LOGO="fedora-logo-icon" 2026-02-19T20:02:01.504 INFO:teuthology.orchestra.run.trial080.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-19T20:02:01.504 INFO:teuthology.orchestra.run.trial080.stdout:HOME_URL="https://centos.org/" 2026-02-19T20:02:01.504 INFO:teuthology.orchestra.run.trial080.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-19T20:02:01.504 INFO:teuthology.orchestra.run.trial080.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-19T20:02:01.504 INFO:teuthology.orchestra.run.trial080.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-19T20:02:01.504 INFO:teuthology.lock.ops:Updating trial080.front.sepia.ceph.com on lock server 2026-02-19T20:02:01.585 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-19T20:02:01.598 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-19T20:02:01.609 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-19T20:02:01.610 DEBUG:teuthology.orchestra.run.trial073:> test '!' -e /home/ubuntu/cephtest 2026-02-19T20:02:01.611 DEBUG:teuthology.orchestra.run.trial080:> test '!' -e /home/ubuntu/cephtest 2026-02-19T20:02:01.626 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-19T20:02:01.637 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-19T20:02:01.637 DEBUG:teuthology.orchestra.run.trial073:> test -z $(ls -A /var/lib/ceph) 2026-02-19T20:02:01.668 DEBUG:teuthology.orchestra.run.trial080:> test -z $(ls -A /var/lib/ceph) 2026-02-19T20:02:01.685 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-19T20:02:01.845 INFO:teuthology.run_tasks:Running task kernel... 2026-02-19T20:02:01.865 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-19T20:02:01.865 INFO:teuthology.task.kernel:config {'cluster1.mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'cluster2.mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-19T20:02:01.865 DEBUG:teuthology.orchestra.run.trial073:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-19T20:02:01.865 DEBUG:teuthology.orchestra.run.trial080:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-19T20:02:01.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T20:02:01.880 DEBUG:teuthology.orchestra.run.trial073:> uname -r 2026-02-19T20:02:01.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T20:02:01.880 DEBUG:teuthology.orchestra.run.trial080:> uname -r 2026-02-19T20:02:01.934 INFO:teuthology.orchestra.run.trial073.stdout:5.14.0-681.el9.x86_64 2026-02-19T20:02:01.935 INFO:teuthology.task.kernel:Running kernel on trial073: 5.14.0-681.el9.x86_64 2026-02-19T20:02:01.935 DEBUG:teuthology.orchestra.run.trial073:> sudo yum install -y kernel 2026-02-19T20:02:01.936 INFO:teuthology.orchestra.run.trial080.stdout:5.14.0-681.el9.x86_64 2026-02-19T20:02:01.936 INFO:teuthology.task.kernel:Running kernel on trial080: 5.14.0-681.el9.x86_64 2026-02-19T20:02:01.936 DEBUG:teuthology.orchestra.run.trial080:> sudo yum install -y kernel 2026-02-19T20:02:03.054 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.9 MB 00:00 2026-02-19T20:02:12.035 INFO:teuthology.orchestra.run.trial073.stdout:CentOS Stream 9 - BaseOS 937 kB/s | 8.9 MB 00:09 2026-02-19T20:02:30.465 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:26 2026-02-19T20:02:33.794 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 7.8 MB 00:01 2026-02-19T20:02:35.006 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 20 kB 00:00 2026-02-19T20:02:35.597 INFO:teuthology.orchestra.run.trial080.stdout:Extra Packages for Enterprise Linux 38 MB/s | 20 MB 00:00 2026-02-19T20:02:38.773 INFO:teuthology.orchestra.run.trial080.stdout:lab-extras 1.9 MB/s | 50 kB 00:00 2026-02-19T20:02:39.131 INFO:teuthology.orchestra.run.trial073.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:26 2026-02-19T20:02:39.780 INFO:teuthology.orchestra.run.trial080.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-19T20:02:39.781 INFO:teuthology.orchestra.run.trial080.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-19T20:02:39.799 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:02:39.800 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:02:39.800 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:02:39.824 DEBUG:teuthology.orchestra.run.trial080:> echo no | sudo yum reinstall kernel || true 2026-02-19T20:02:40.120 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:02 ago on Thu 19 Feb 2026 08:02:38 PM UTC. 2026-02-19T20:02:40.200 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:02:40.201 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout: Package Architecture Version Repository Size 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout:Reinstalling: 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 803 k 2026-02-19T20:02:40.202 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 0 2026-02-19T20:02:40.203 INFO:teuthology.orchestra.run.trial080.stderr:Operation aborted. 2026-02-19T20:02:40.221 INFO:teuthology.orchestra.run.trial080.stdout:Is this ok [y/N]: 2026-02-19T20:02:40.221 DEBUG:teuthology.orchestra.run.trial080:> sudo yum reinstall -y kernel || true 2026-02-19T20:02:40.524 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:02 ago on Thu 19 Feb 2026 08:02:38 PM UTC. 2026-02-19T20:02:40.603 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout: Package Architecture Version Repository Size 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout:Reinstalling: 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 803 k 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 0 2026-02-19T20:02:40.604 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:02:40.774 INFO:teuthology.orchestra.run.trial080.stdout:kernel-5.14.0-681.el9.x86_64.rpm 13 MB/s | 803 kB 00:00 2026-02-19T20:02:40.774 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:02:40.775 INFO:teuthology.orchestra.run.trial080.stdout:Total 4.6 MB/s | 803 kB 00:00 2026-02-19T20:02:40.785 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:02:40.818 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:02:40.818 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:02:40.823 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:02:40.823 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:02:40.847 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:02:40.850 INFO:teuthology.orchestra.run.trial080.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-19T20:02:40.851 INFO:teuthology.orchestra.run.trial080.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-19T20:02:40.957 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-19T20:02:40.958 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-19T20:02:41.013 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-19T20:02:41.014 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:02:41.014 INFO:teuthology.orchestra.run.trial080.stdout:Reinstalled: 2026-02-19T20:02:41.014 INFO:teuthology.orchestra.run.trial080.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-19T20:02:41.014 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:02:41.014 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:02:41.040 DEBUG:teuthology.orchestra.run.trial080:> rpm -q kernel | sort -rV | head -n 1 2026-02-19T20:02:41.064 INFO:teuthology.orchestra.run.trial080.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-19T20:02:41.065 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-19T20:02:41.066 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-19T20:02:41.066 INFO:teuthology.task.kernel:Enabling kdb on cluster2.mon.a... 2026-02-19T20:02:41.066 DEBUG:teuthology.orchestra.run.trial080:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-19T20:02:41.130 INFO:teuthology.orchestra.run.trial080.stdout:ttyS1 2026-02-19T20:02:41.162 DEBUG:teuthology.parallel:result is None 2026-02-19T20:02:43.054 INFO:teuthology.orchestra.run.trial073.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 7.8 MB 00:01 2026-02-19T20:02:43.877 INFO:teuthology.orchestra.run.trial073.stdout:CentOS Stream 9 - Extras packages 192 kB/s | 20 kB 00:00 2026-02-19T20:02:44.862 INFO:teuthology.orchestra.run.trial073.stdout:Extra Packages for Enterprise Linux 22 MB/s | 20 MB 00:00 2026-02-19T20:02:48.070 INFO:teuthology.orchestra.run.trial073.stdout:lab-extras 1.4 MB/s | 50 kB 00:00 2026-02-19T20:02:49.077 INFO:teuthology.orchestra.run.trial073.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-19T20:02:49.077 INFO:teuthology.orchestra.run.trial073.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-19T20:02:49.096 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:02:49.097 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:02:49.097 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:02:49.121 DEBUG:teuthology.orchestra.run.trial073:> echo no | sudo yum reinstall kernel || true 2026-02-19T20:02:49.421 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:01 ago on Thu 19 Feb 2026 08:02:48 PM UTC. 2026-02-19T20:02:49.504 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:02:49.505 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:02:49.505 INFO:teuthology.orchestra.run.trial073.stdout: Package Architecture Version Repository Size 2026-02-19T20:02:49.505 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:02:49.505 INFO:teuthology.orchestra.run.trial073.stdout:Reinstalling: 2026-02-19T20:02:49.505 INFO:teuthology.orchestra.run.trial073.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-19T20:02:49.505 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:02:49.505 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:02:49.506 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:02:49.506 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:02:49.506 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 803 k 2026-02-19T20:02:49.506 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 0 2026-02-19T20:02:49.506 INFO:teuthology.orchestra.run.trial073.stderr:Operation aborted. 2026-02-19T20:02:49.525 INFO:teuthology.orchestra.run.trial073.stdout:Is this ok [y/N]: 2026-02-19T20:02:49.526 DEBUG:teuthology.orchestra.run.trial073:> sudo yum reinstall -y kernel || true 2026-02-19T20:02:49.824 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:01 ago on Thu 19 Feb 2026 08:02:48 PM UTC. 2026-02-19T20:02:49.905 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:02:49.905 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout: Package Architecture Version Repository Size 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout:Reinstalling: 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 803 k 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 0 2026-02-19T20:02:49.906 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:02:50.229 INFO:teuthology.orchestra.run.trial073.stdout:kernel-5.14.0-681.el9.x86_64.rpm 13 MB/s | 803 kB 00:00 2026-02-19T20:02:50.229 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:02:50.229 INFO:teuthology.orchestra.run.trial073.stdout:Total 2.4 MB/s | 803 kB 00:00 2026-02-19T20:02:50.239 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:02:50.275 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:02:50.275 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:02:50.279 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:02:50.280 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:02:50.304 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:02:50.307 INFO:teuthology.orchestra.run.trial073.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-19T20:02:50.308 INFO:teuthology.orchestra.run.trial073.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-19T20:02:50.420 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-19T20:02:50.420 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-19T20:02:50.472 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-19T20:02:50.472 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:02:50.472 INFO:teuthology.orchestra.run.trial073.stdout:Reinstalled: 2026-02-19T20:02:50.472 INFO:teuthology.orchestra.run.trial073.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-19T20:02:50.472 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:02:50.473 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:02:50.501 DEBUG:teuthology.orchestra.run.trial073:> rpm -q kernel | sort -rV | head -n 1 2026-02-19T20:02:50.524 INFO:teuthology.orchestra.run.trial073.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-19T20:02:50.525 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-19T20:02:50.525 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-19T20:02:50.525 INFO:teuthology.task.kernel:Enabling kdb on cluster1.mon.a... 2026-02-19T20:02:50.525 DEBUG:teuthology.orchestra.run.trial073:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-19T20:02:50.587 INFO:teuthology.orchestra.run.trial073.stdout:ttyS1 2026-02-19T20:02:50.607 DEBUG:teuthology.parallel:result is None 2026-02-19T20:02:50.607 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-19T20:02:50.619 INFO:teuthology.task.internal:Creating test directory... 2026-02-19T20:02:50.619 DEBUG:teuthology.orchestra.run.trial073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-19T20:02:50.649 DEBUG:teuthology.orchestra.run.trial080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-19T20:02:50.665 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-19T20:02:50.677 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-19T20:02:50.689 INFO:teuthology.task.internal:Creating archive directory... 2026-02-19T20:02:50.690 DEBUG:teuthology.orchestra.run.trial073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-19T20:02:50.705 DEBUG:teuthology.orchestra.run.trial080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-19T20:02:50.731 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-19T20:02:50.743 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-19T20:02:50.743 DEBUG:teuthology.orchestra.run.trial073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-19T20:02:50.763 DEBUG:teuthology.orchestra.run.trial080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-19T20:02:50.784 INFO:teuthology.orchestra.run.trial073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T20:02:50.787 INFO:teuthology.orchestra.run.trial080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T20:02:50.793 INFO:teuthology.orchestra.run.trial073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T20:02:50.797 INFO:teuthology.orchestra.run.trial080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T20:02:50.798 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-19T20:02:50.810 INFO:teuthology.task.internal:Configuring sudo... 2026-02-19T20:02:50.810 DEBUG:teuthology.orchestra.run.trial073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-19T20:02:50.837 DEBUG:teuthology.orchestra.run.trial080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-19T20:02:50.863 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-19T20:02:50.880 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-19T20:02:50.880 DEBUG:teuthology.orchestra.run.trial073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-19T20:02:50.901 DEBUG:teuthology.orchestra.run.trial080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-19T20:02:50.919 DEBUG:teuthology.orchestra.run.trial073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T20:02:51.080 DEBUG:teuthology.orchestra.run.trial073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T20:02:51.107 DEBUG:teuthology.orchestra.run.trial073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T20:02:51.252 DEBUG:teuthology.orchestra.run.trial073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T20:02:51.277 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:02:51.277 DEBUG:teuthology.orchestra.run.trial073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-19T20:02:51.340 DEBUG:teuthology.orchestra.run.trial080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T20:02:51.455 DEBUG:teuthology.orchestra.run.trial080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T20:02:51.481 DEBUG:teuthology.orchestra.run.trial080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T20:02:51.618 DEBUG:teuthology.orchestra.run.trial080:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T20:02:51.641 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:02:51.641 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-19T20:02:51.704 DEBUG:teuthology.orchestra.run.trial073:> sudo service rsyslog restart 2026-02-19T20:02:51.706 DEBUG:teuthology.orchestra.run.trial080:> sudo service rsyslog restart 2026-02-19T20:02:51.733 INFO:teuthology.orchestra.run.trial073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-19T20:02:51.774 INFO:teuthology.orchestra.run.trial080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-19T20:02:52.192 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-19T20:02:52.207 INFO:teuthology.task.internal:Starting timer... 2026-02-19T20:02:52.208 INFO:teuthology.run_tasks:Running task pcp... 2026-02-19T20:02:52.232 INFO:teuthology.run_tasks:Running task selinux... 2026-02-19T20:02:52.252 DEBUG:teuthology.orchestra.run.trial073:> sudo service auditd rotate 2026-02-19T20:02:52.281 INFO:teuthology.orchestra.run.trial073.stdout:Rotating logs: 2026-02-19T20:02:52.282 DEBUG:teuthology.orchestra.run.trial080:> sudo service auditd rotate 2026-02-19T20:02:52.312 INFO:teuthology.orchestra.run.trial080.stdout:Rotating logs: 2026-02-19T20:02:52.314 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-19T20:02:52.314 DEBUG:teuthology.orchestra.run.trial073:> /usr/sbin/getenforce 2026-02-19T20:02:52.338 INFO:teuthology.orchestra.run.trial073.stdout:Permissive 2026-02-19T20:02:52.338 DEBUG:teuthology.orchestra.run.trial080:> /usr/sbin/getenforce 2026-02-19T20:02:52.370 INFO:teuthology.orchestra.run.trial080.stdout:Permissive 2026-02-19T20:02:52.370 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial073.front.sepia.ceph.com': 'permissive', 'ubuntu@trial080.front.sepia.ceph.com': 'permissive'} 2026-02-19T20:02:52.371 DEBUG:teuthology.orchestra.run.trial073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-19T20:02:52.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T20:02:52.403 DEBUG:teuthology.orchestra.run.trial080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-19T20:02:52.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T20:02:52.435 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-19T20:02:52.435 DEBUG:teuthology.orchestra.run.trial073:> sudo /usr/sbin/setenforce permissive 2026-02-19T20:02:52.467 DEBUG:teuthology.orchestra.run.trial080:> sudo /usr/sbin/setenforce permissive 2026-02-19T20:02:52.498 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-19T20:02:52.514 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-19T20:02:52.517 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-19T20:02:52.622 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-19T20:02:52.635 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-19T20:02:52.637 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial073.front.sepia.ceph.com,trial080.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-19T20:05:36.102 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial073.front.sepia.ceph.com'), Remote(name='ubuntu@trial080.front.sepia.ceph.com')] 2026-02-19T20:05:36.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial073.front.sepia.ceph.com' 2026-02-19T20:05:36.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T20:05:36.210 DEBUG:teuthology.orchestra.run.trial073:> true 2026-02-19T20:05:36.292 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial073.front.sepia.ceph.com' 2026-02-19T20:05:36.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial080.front.sepia.ceph.com' 2026-02-19T20:05:36.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T20:05:36.376 DEBUG:teuthology.orchestra.run.trial080:> true 2026-02-19T20:05:36.453 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial080.front.sepia.ceph.com' 2026-02-19T20:05:36.453 INFO:teuthology.run_tasks:Running task clock... 2026-02-19T20:05:36.466 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-19T20:05:36.467 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-19T20:05:36.467 DEBUG:teuthology.orchestra.run.trial073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T20:05:36.469 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-19T20:05:36.470 DEBUG:teuthology.orchestra.run.trial080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T20:05:36.496 INFO:teuthology.orchestra.run.trial073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-19T20:05:36.508 INFO:teuthology.orchestra.run.trial073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-19T20:05:36.520 INFO:teuthology.orchestra.run.trial080.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-19T20:05:36.528 INFO:teuthology.orchestra.run.trial073.stderr:sudo: ntpd: command not found 2026-02-19T20:05:36.533 INFO:teuthology.orchestra.run.trial080.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-19T20:05:36.538 INFO:teuthology.orchestra.run.trial073.stdout:506 Cannot talk to daemon 2026-02-19T20:05:36.550 INFO:teuthology.orchestra.run.trial073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-19T20:05:36.553 INFO:teuthology.orchestra.run.trial080.stderr:sudo: ntpd: command not found 2026-02-19T20:05:36.562 INFO:teuthology.orchestra.run.trial073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-19T20:05:36.563 INFO:teuthology.orchestra.run.trial080.stdout:506 Cannot talk to daemon 2026-02-19T20:05:36.574 INFO:teuthology.orchestra.run.trial080.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-19T20:05:36.587 INFO:teuthology.orchestra.run.trial080.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-19T20:05:36.613 INFO:teuthology.orchestra.run.trial073.stderr:bash: line 1: ntpq: command not found 2026-02-19T20:05:36.615 INFO:teuthology.orchestra.run.trial073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-19T20:05:36.615 INFO:teuthology.orchestra.run.trial073.stdout:=============================================================================== 2026-02-19T20:05:36.630 INFO:teuthology.orchestra.run.trial080.stderr:bash: line 1: ntpq: command not found 2026-02-19T20:05:36.631 INFO:teuthology.orchestra.run.trial080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-19T20:05:36.632 INFO:teuthology.orchestra.run.trial080.stdout:=============================================================================== 2026-02-19T20:05:36.633 INFO:teuthology.run_tasks:Running task install... 2026-02-19T20:05:36.646 DEBUG:teuthology.task.install:project ceph 2026-02-19T20:05:36.647 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}} 2026-02-19T20:05:36.647 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'} 2026-02-19T20:05:36.647 INFO:teuthology.task.install:Using flavor: default 2026-02-19T20:05:36.656 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-19T20:05:36.656 INFO:teuthology.task.install:extra packages: ['rbd-mirror'] 2026-02-19T20:05:36.657 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-mirror'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a', 'tag': None, 'wait_for_package': False} 2026-02-19T20:05:36.658 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T20:05:36.660 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-mirror'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a', 'tag': None, 'wait_for_package': False} 2026-02-19T20:05:36.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T20:05:36.852 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/ 2026-02-19T20:05:36.853 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5336.g802bf111 2026-02-19T20:05:36.862 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/ 2026-02-19T20:05:36.862 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5336.g802bf111 2026-02-19T20:05:37.003 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-19T20:05:37.004 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:05:37.004 DEBUG:teuthology.orchestra.run.trial073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-19T20:05:37.011 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2026-02-18-1820/802bf111cd55e57f5842153d7ba2b8b964aa818a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-19T20:05:37.012 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:05:37.012 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-19T20:05:37.029 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-mirror on remote rpm x86_64 2026-02-19T20:05:37.029 DEBUG:teuthology.orchestra.run.trial073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/802bf111cd55e57f5842153d7ba2b8b964aa818a/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-19T20:05:37.037 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-mirror on remote rpm x86_64 2026-02-19T20:05:37.037 DEBUG:teuthology.orchestra.run.trial080:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/802bf111cd55e57f5842153d7ba2b8b964aa818a/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-19T20:05:37.095 DEBUG:teuthology.orchestra.run.trial073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-19T20:05:37.102 DEBUG:teuthology.orchestra.run.trial080:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-19T20:05:37.170 DEBUG:teuthology.orchestra.run.trial073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-19T20:05:37.175 DEBUG:teuthology.orchestra.run.trial080:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-19T20:05:37.233 INFO:teuthology.orchestra.run.trial073.stdout:check_obsoletes = 1 2026-02-19T20:05:37.235 DEBUG:teuthology.orchestra.run.trial073:> sudo yum clean all 2026-02-19T20:05:37.239 INFO:teuthology.orchestra.run.trial080.stdout:check_obsoletes = 1 2026-02-19T20:05:37.240 DEBUG:teuthology.orchestra.run.trial080:> sudo yum clean all 2026-02-19T20:05:37.393 INFO:teuthology.orchestra.run.trial073.stdout:41 files removed 2026-02-19T20:05:37.398 INFO:teuthology.orchestra.run.trial080.stdout:41 files removed 2026-02-19T20:05:37.408 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-radosgw 2026-02-19T20:05:37.413 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-radosgw 2026-02-19T20:05:37.769 INFO:teuthology.orchestra.run.trial073.stdout:ceph packages for x86_64 431 kB/s | 93 kB 00:00 2026-02-19T20:05:37.818 INFO:teuthology.orchestra.run.trial080.stdout:ceph packages for x86_64 356 kB/s | 93 kB 00:00 2026-02-19T20:05:37.910 INFO:teuthology.orchestra.run.trial073.stdout:ceph noarch packages 213 kB/s | 24 kB 00:00 2026-02-19T20:05:37.962 INFO:teuthology.orchestra.run.trial080.stdout:ceph noarch packages 206 kB/s | 24 kB 00:00 2026-02-19T20:05:38.040 INFO:teuthology.orchestra.run.trial073.stdout:ceph source packages 19 kB/s | 2.1 kB 00:00 2026-02-19T20:05:38.094 INFO:teuthology.orchestra.run.trial080.stdout:ceph source packages 19 kB/s | 2.1 kB 00:00 2026-02-19T20:05:38.770 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.9 MB 00:00 2026-02-19T20:05:43.118 INFO:teuthology.orchestra.run.trial073.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.9 MB 00:05 2026-02-19T20:05:44.092 INFO:teuthology.orchestra.run.trial073.stdout:CentOS Stream 9 - AppStream 60 MB/s | 27 MB 00:00 2026-02-19T20:05:46.949 INFO:teuthology.orchestra.run.trial073.stdout:CentOS Stream 9 - CRB 16 MB/s | 7.8 MB 00:00 2026-02-19T20:05:47.826 INFO:teuthology.orchestra.run.trial073.stdout:CentOS Stream 9 - Extras packages 124 kB/s | 20 kB 00:00 2026-02-19T20:05:49.154 INFO:teuthology.orchestra.run.trial073.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2026-02-19T20:05:52.389 INFO:teuthology.orchestra.run.trial073.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-19T20:05:53.580 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:05:53.581 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:05:53.582 INFO:teuthology.orchestra.run.trial073.stdout: Package Arch Version Repository Size 2026-02-19T20:05:53.582 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:05:53.582 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:05:53.582 INFO:teuthology.orchestra.run.trial073.stdout: ceph-radosgw x86_64 2:20.3.0-5336.g802bf111.el9 ceph 13 M 2026-02-19T20:05:53.582 INFO:teuthology.orchestra.run.trial073.stdout:Upgrading: 2026-02-19T20:05:53.582 INFO:teuthology.orchestra.run.trial073.stdout: librados2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 3.6 M 2026-02-19T20:05:53.582 INFO:teuthology.orchestra.run.trial073.stdout: librbd1 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 2.8 M 2026-02-19T20:05:53.582 INFO:teuthology.orchestra.run.trial073.stdout:Installing dependencies: 2026-02-19T20:05:53.582 INFO:teuthology.orchestra.run.trial073.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: ceph-base x86_64 2:20.3.0-5336.g802bf111.el9 ceph 5.2 M 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: ceph-common x86_64 2:20.3.0-5336.g802bf111.el9 ceph 24 M 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: ceph-selinux x86_64 2:20.3.0-5336.g802bf111.el9 ceph 25 k 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: libcephfs2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 910 k 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-19T20:05:53.583 INFO:teuthology.orchestra.run.trial073.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: libradosstriper1 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 250 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: librgw2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 6.5 M 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: python3-ceph-argparse x86_64 2:20.3.0-5336.g802bf111.el9 ceph 45 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: python3-ceph-common x86_64 2:20.3.0-5336.g802bf111.el9 ceph 185 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: python3-cephfs x86_64 2:20.3.0-5336.g802bf111.el9 ceph 203 k 2026-02-19T20:05:53.584 INFO:teuthology.orchestra.run.trial073.stdout: python3-rados x86_64 2:20.3.0-5336.g802bf111.el9 ceph 323 k 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: python3-rbd x86_64 2:20.3.0-5336.g802bf111.el9 ceph 305 k 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: python3-rgw x86_64 2:20.3.0-5336.g802bf111.el9 ceph 100 k 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout:Installing weak dependencies: 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: libcephfs-daemon x86_64 2:20.3.0-5336.g802bf111.el9 ceph 37 k 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 25 k 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:05:53.585 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:05:53.586 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:05:53.586 INFO:teuthology.orchestra.run.trial073.stdout:Install 35 Packages 2026-02-19T20:05:53.586 INFO:teuthology.orchestra.run.trial073.stdout:Upgrade 2 Packages 2026-02-19T20:05:53.586 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:05:53.586 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 68 M 2026-02-19T20:05:53.587 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:05:54.081 INFO:teuthology.orchestra.run.trial073.stdout:(1/37): ceph-base-20.3.0-5336.g802bf111.el9.x86 18 MB/s | 5.2 MB 00:00 2026-02-19T20:05:54.115 INFO:teuthology.orchestra.run.trial073.stdout:(2/37): ceph-selinux-20.3.0-5336.g802bf111.el9. 733 kB/s | 25 kB 00:00 2026-02-19T20:05:54.151 INFO:teuthology.orchestra.run.trial073.stdout:(3/37): libcephfs-daemon-20.3.0-5336.g802bf111. 1.0 MB/s | 37 kB 00:00 2026-02-19T20:05:54.191 INFO:teuthology.orchestra.run.trial073.stdout:(4/37): libcephfs-proxy2-20.3.0-5336.g802bf111. 660 kB/s | 25 kB 00:00 2026-02-19T20:05:54.249 INFO:teuthology.orchestra.run.trial073.stdout:(5/37): libcephfs2-20.3.0-5336.g802bf111.el9.x8 15 MB/s | 910 kB 00:00 2026-02-19T20:05:54.296 INFO:teuthology.orchestra.run.trial073.stdout:(6/37): libradosstriper1-20.3.0-5336.g802bf111. 5.2 MB/s | 250 kB 00:00 2026-02-19T20:05:54.552 INFO:teuthology.orchestra.run.trial073.stdout:(7/37): ceph-radosgw-20.3.0-5336.g802bf111.el9. 17 MB/s | 13 MB 00:00 2026-02-19T20:05:54.587 INFO:teuthology.orchestra.run.trial073.stdout:(8/37): python3-ceph-argparse-20.3.0-5336.g802b 1.3 MB/s | 45 kB 00:00 2026-02-19T20:05:54.626 INFO:teuthology.orchestra.run.trial073.stdout:(9/37): librgw2-20.3.0-5336.g802bf111.el9.x86_6 20 MB/s | 6.5 MB 00:00 2026-02-19T20:05:54.627 INFO:teuthology.orchestra.run.trial073.stdout:(10/37): python3-ceph-common-20.3.0-5336.g802bf 4.5 MB/s | 185 kB 00:00 2026-02-19T20:05:54.649 INFO:teuthology.orchestra.run.trial073.stdout:(11/37): python3-cephfs-20.3.0-5336.g802bf111.e 8.8 MB/s | 203 kB 00:00 2026-02-19T20:05:54.656 INFO:teuthology.orchestra.run.trial073.stdout:(12/37): python3-rados-20.3.0-5336.g802bf111.el 11 MB/s | 323 kB 00:00 2026-02-19T20:05:54.680 INFO:teuthology.orchestra.run.trial073.stdout:(13/37): python3-rbd-20.3.0-5336.g802bf111.el9. 9.7 MB/s | 305 kB 00:00 2026-02-19T20:05:54.682 INFO:teuthology.orchestra.run.trial073.stdout:(14/37): python3-rgw-20.3.0-5336.g802bf111.el9. 3.7 MB/s | 100 kB 00:00 2026-02-19T20:05:54.706 INFO:teuthology.orchestra.run.trial073.stdout:(15/37): mailcap-2.1.49-5.el9.noarch.rpm 1.2 MB/s | 33 kB 00:00 2026-02-19T20:05:54.724 INFO:teuthology.orchestra.run.trial073.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 5.4 MB/s | 93 kB 00:00 2026-02-19T20:05:54.734 INFO:teuthology.orchestra.run.trial073.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 10 MB/s | 104 kB 00:00 2026-02-19T20:05:54.744 INFO:teuthology.orchestra.run.trial073.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 15 MB/s | 164 kB 00:00 2026-02-19T20:05:54.746 INFO:teuthology.orchestra.run.trial073.stdout:(19/37): nvme-cli-2.16-1.el9.x86_64.rpm 18 MB/s | 1.2 MB 00:00 2026-02-19T20:05:54.783 INFO:teuthology.orchestra.run.trial073.stdout:(20/37): ceph-common-20.3.0-5336.g802bf111.el9. 24 MB/s | 24 MB 00:00 2026-02-19T20:05:54.784 INFO:teuthology.orchestra.run.trial073.stdout:(21/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.2 MB/s | 45 kB 00:00 2026-02-19T20:05:54.784 INFO:teuthology.orchestra.run.trial073.stdout:(22/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.9 MB/s | 160 kB 00:00 2026-02-19T20:05:54.794 INFO:teuthology.orchestra.run.trial073.stdout:(23/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 29 MB/s | 292 kB 00:00 2026-02-19T20:05:54.796 INFO:teuthology.orchestra.run.trial073.stdout:(24/37): qatlib-25.08.0-2.el9.x86_64.rpm 20 MB/s | 240 kB 00:00 2026-02-19T20:05:54.802 INFO:teuthology.orchestra.run.trial073.stdout:(25/37): qatlib-service-25.08.0-2.el9.x86_64.rp 4.2 MB/s | 37 kB 00:00 2026-02-19T20:05:54.805 INFO:teuthology.orchestra.run.trial073.stdout:(26/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 7.4 MB/s | 66 kB 00:00 2026-02-19T20:05:54.838 INFO:teuthology.orchestra.run.trial073.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 8.4 MB/s | 308 kB 00:00 2026-02-19T20:05:54.840 INFO:teuthology.orchestra.run.trial073.stdout:(28/37): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2026-02-19T20:05:54.843 INFO:teuthology.orchestra.run.trial073.stdout:(29/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 5.1 MB/s | 25 kB 00:00 2026-02-19T20:05:54.849 INFO:teuthology.orchestra.run.trial073.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 12 MB/s | 67 kB 00:00 2026-02-19T20:05:54.855 INFO:teuthology.orchestra.run.trial073.stdout:(31/37): libarrow-9.0.0-15.el9.x86_64.rpm 88 MB/s | 4.4 MB 00:00 2026-02-19T20:05:54.855 INFO:teuthology.orchestra.run.trial073.stdout:(32/37): liboath-2.6.12-1.el9.x86_64.rpm 3.1 MB/s | 49 kB 00:00 2026-02-19T20:05:54.862 INFO:teuthology.orchestra.run.trial073.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 28 MB/s | 191 kB 00:00 2026-02-19T20:05:54.863 INFO:teuthology.orchestra.run.trial073.stdout:(34/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 58 MB/s | 838 kB 00:00 2026-02-19T20:05:54.881 INFO:teuthology.orchestra.run.trial073.stdout:(35/37): thrift-0.15.0-4.el9.x86_64.rpm 63 MB/s | 1.6 MB 00:00 2026-02-19T20:05:54.970 INFO:teuthology.orchestra.run.trial073.stdout:(36/37): librbd1-20.3.0-5336.g802bf111.el9.x86_ 26 MB/s | 2.8 MB 00:00 2026-02-19T20:05:54.978 INFO:teuthology.orchestra.run.trial073.stdout:(37/37): librados2-20.3.0-5336.g802bf111.el9.x8 31 MB/s | 3.6 MB 00:00 2026-02-19T20:05:54.979 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:05:54.980 INFO:teuthology.orchestra.run.trial073.stdout:Total 49 MB/s | 68 MB 00:01 2026-02-19T20:05:55.080 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:05:55.098 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:05:55.098 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:05:55.369 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:05:55.369 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:05:55.681 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:05:55.839 INFO:teuthology.orchestra.run.trial073.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-19T20:05:55.843 INFO:teuthology.orchestra.run.trial073.stdout: Upgrading : librados2-2:20.3.0-5336.g802bf111.el9.x86_64 2/39 2026-02-19T20:05:55.891 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: librados2-2:20.3.0-5336.g802bf111.el9.x86_64 2/39 2026-02-19T20:05:55.892 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 3/39 2026-02-19T20:05:55.905 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 3/39 2026-02-19T20:05:55.907 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libcephfs-daemon-2:20.3.0-5336.g802bf111.el9.x86_6 4/39 2026-02-19T20:05:55.908 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_6 5/39 2026-02-19T20:05:55.934 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_6 5/39 2026-02-19T20:05:55.937 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 6/39 2026-02-19T20:05:55.946 INFO:teuthology.orchestra.run.trial073.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-19T20:05:55.948 INFO:teuthology.orchestra.run.trial073.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-19T20:05:55.951 INFO:teuthology.orchestra.run.trial073.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-19T20:05:55.955 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-19T20:05:55.967 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-ceph-argparse-2:20.3.0-5336.g802bf111.el9. 11/39 2026-02-19T20:05:55.978 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 12/39 2026-02-19T20:05:55.979 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_6 13/39 2026-02-19T20:05:56.000 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_6 13/39 2026-02-19T20:05:56.003 INFO:teuthology.orchestra.run.trial073.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-19T20:05:56.036 INFO:teuthology.orchestra.run.trial073.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-19T20:05:56.041 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-19T20:05:56.047 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-19T20:05:56.049 INFO:teuthology.orchestra.run.trial073.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-19T20:05:56.054 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-19T20:05:56.178 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-19T20:05:56.179 INFO:teuthology.orchestra.run.trial073.stdout: Upgrading : librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 21/39 2026-02-19T20:05:56.205 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 21/39 2026-02-19T20:05:56.207 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 22/39 2026-02-19T20:05:56.213 INFO:teuthology.orchestra.run.trial073.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-19T20:05:56.496 INFO:teuthology.orchestra.run.trial073.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-19T20:05:56.498 INFO:teuthology.orchestra.run.trial073.stdout: Installing : librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 25/39 2026-02-19T20:05:56.514 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 25/39 2026-02-19T20:05:56.519 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 26/39 2026-02-19T20:05:56.525 INFO:teuthology.orchestra.run.trial073.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-19T20:05:56.528 INFO:teuthology.orchestra.run.trial073.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-19T20:05:56.529 INFO:teuthology.orchestra.run.trial073.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-19T20:05:56.547 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-19T20:05:56.661 INFO:teuthology.orchestra.run.trial073.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-19T20:05:56.663 INFO:teuthology.orchestra.run.trial073.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-19T20:05:56.943 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-19T20:05:56.943 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-19T20:05:56.943 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:05:56.987 INFO:teuthology.orchestra.run.trial073.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-19T20:05:56.990 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-ceph-common-2:20.3.0-5336.g802bf111.el9.x8 33/39 2026-02-19T20:05:58.086 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 34/39 2026-02-19T20:05:58.088 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 34/39 2026-02-19T20:05:58.349 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 34/39 2026-02-19T20:05:58.353 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 35/39 2026-02-19T20:05:58.395 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 35/39 2026-02-19T20:05:58.395 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-19T20:05:58.396 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-19T20:05:58.396 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:05:58.399 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 36/39 2026-02-19T20:06:02.903 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 36/39 2026-02-19T20:06:02.903 INFO:teuthology.orchestra.run.trial073.stdout:skipping the directory /sys 2026-02-19T20:06:02.903 INFO:teuthology.orchestra.run.trial073.stdout:skipping the directory /proc 2026-02-19T20:06:02.903 INFO:teuthology.orchestra.run.trial073.stdout:skipping the directory /mnt 2026-02-19T20:06:02.903 INFO:teuthology.orchestra.run.trial073.stdout:skipping the directory /var/tmp 2026-02-19T20:06:02.903 INFO:teuthology.orchestra.run.trial073.stdout:skipping the directory /home 2026-02-19T20:06:02.903 INFO:teuthology.orchestra.run.trial073.stdout:skipping the directory /root 2026-02-19T20:06:02.903 INFO:teuthology.orchestra.run.trial073.stdout:skipping the directory /tmp 2026-02-19T20:06:02.903 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:03.482 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 37/39 2026-02-19T20:06:03.505 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 37/39 2026-02-19T20:06:03.506 INFO:teuthology.orchestra.run.trial073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:03.506 INFO:teuthology.orchestra.run.trial073.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-19T20:06:03.506 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-19T20:06:03.506 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-19T20:06:03.506 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:03.507 INFO:teuthology.orchestra.run.trial073.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-19T20:06:03.520 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-19T20:06:03.520 INFO:teuthology.orchestra.run.trial073.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-19T20:06:03.575 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-19T20:06:03.576 INFO:teuthology.orchestra.run.trial073.stdout:Creating group 'qat' with GID 992. 2026-02-19T20:06:03.576 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:03.883 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 1/39 2026-02-19T20:06:03.883 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 2/39 2026-02-19T20:06:03.883 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 3/39 2026-02-19T20:06:03.883 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 4/39 2026-02-19T20:06:03.883 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libcephfs-daemon-2:20.3.0-5336.g802bf111.el9.x86_6 5/39 2026-02-19T20:06:03.883 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_6 6/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 7/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_6 8/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 9/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5336.g802bf111.el9. 10/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-ceph-common-2:20.3.0-5336.g802bf111.el9.x8 11/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 12/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 13/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 14/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 15/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-19T20:06:03.884 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-19T20:06:03.885 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-19T20:06:03.886 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-19T20:06:03.886 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-19T20:06:03.886 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-19T20:06:03.886 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-19T20:06:03.886 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-19T20:06:03.886 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : librados2-2:20.3.0-5336.g802bf111.el9.x86_64 36/39 2026-02-19T20:06:03.886 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-19T20:06:03.886 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 38/39 2026-02-19T20:06:03.944 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-19T20:06:03.944 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:03.944 INFO:teuthology.orchestra.run.trial073.stdout:Upgraded: 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: librados2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-19T20:06:03.945 INFO:teuthology.orchestra.run.trial073.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: libcephfs-daemon-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-19T20:06:03.946 INFO:teuthology.orchestra.run.trial073.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: python3-ceph-argparse-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: python3-ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-19T20:06:03.947 INFO:teuthology.orchestra.run.trial073.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-19T20:06:03.948 INFO:teuthology.orchestra.run.trial073.stdout: re2-1:20211101-20.el9.x86_64 2026-02-19T20:06:03.948 INFO:teuthology.orchestra.run.trial073.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-19T20:06:03.948 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:03.948 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:03.988 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-test 2026-02-19T20:06:04.301 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:12 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:04.384 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:04.385 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:04.385 INFO:teuthology.orchestra.run.trial073.stdout: Package Arch Version Repository Size 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout: ceph-test x86_64 2:20.3.0-5336.g802bf111.el9 ceph 79 M 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout:Installing dependencies: 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout: libcephsqlite x86_64 2:20.3.0-5336.g802bf111.el9 ceph 159 k 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-19T20:06:04.386 INFO:teuthology.orchestra.run.trial073.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout:Install 10 Packages 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 83 M 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 341 M 2026-02-19T20:06:04.387 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:06:04.873 INFO:teuthology.orchestra.run.trial073.stdout:(1/10): hwloc-libs-2.4.1-6.el9.x86_64.rpm 12 MB/s | 2.1 MB 00:00 2026-02-19T20:06:04.894 INFO:teuthology.orchestra.run.trial073.stdout:(2/10): libatomic-11.5.0-14.el9.x86_64.rpm 1.1 MB/s | 23 kB 00:00 2026-02-19T20:06:04.914 INFO:teuthology.orchestra.run.trial073.stdout:(3/10): lksctp-tools-1.0.19-2.el9.x86_64.rpm 4.5 MB/s | 94 kB 00:00 2026-02-19T20:06:04.963 INFO:teuthology.orchestra.run.trial073.stdout:(4/10): libxslt-1.1.34-12.el9.x86_64.rpm 4.6 MB/s | 233 kB 00:00 2026-02-19T20:06:04.996 INFO:teuthology.orchestra.run.trial073.stdout:(5/10): protobuf-3.14.0-17.el9.x86_64.rpm 31 MB/s | 1.0 MB 00:00 2026-02-19T20:06:05.007 INFO:teuthology.orchestra.run.trial073.stdout:(6/10): socat-1.7.4.1-8.el9.x86_64.rpm 28 MB/s | 303 kB 00:00 2026-02-19T20:06:05.016 INFO:teuthology.orchestra.run.trial073.stdout:(7/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 7.0 MB/s | 64 kB 00:00 2026-02-19T20:06:05.105 INFO:teuthology.orchestra.run.trial073.stdout:(8/10): libcephsqlite-20.3.0-5336.g802bf111.el9 388 kB/s | 159 kB 00:00 2026-02-19T20:06:05.292 INFO:teuthology.orchestra.run.trial073.stdout:(9/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 442 kB/s | 122 kB 00:00 2026-02-19T20:06:06.228 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:26 2026-02-19T20:06:09.228 INFO:teuthology.orchestra.run.trial073.stdout:(10/10): ceph-test-20.3.0-5336.g802bf111.el9.x8 17 MB/s | 79 MB 00:04 2026-02-19T20:06:09.230 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:09.230 INFO:teuthology.orchestra.run.trial073.stdout:Total 17 MB/s | 83 MB 00:04 2026-02-19T20:06:09.300 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:06:09.311 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:06:09.311 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:06:09.453 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 7.8 MB 00:00 2026-02-19T20:06:09.596 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:06:09.596 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:06:09.905 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:06:09.914 INFO:teuthology.orchestra.run.trial073.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2026-02-19T20:06:09.922 INFO:teuthology.orchestra.run.trial073.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/10 2026-02-19T20:06:09.928 INFO:teuthology.orchestra.run.trial073.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/10 2026-02-19T20:06:09.931 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/10 2026-02-19T20:06:09.937 INFO:teuthology.orchestra.run.trial073.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2026-02-19T20:06:09.939 INFO:teuthology.orchestra.run.trial073.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 6/10 2026-02-19T20:06:09.949 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 7/10 2026-02-19T20:06:09.957 INFO:teuthology.orchestra.run.trial073.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 8/10 2026-02-19T20:06:09.958 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 9/10 2026-02-19T20:06:10.407 INFO:teuthology.orchestra.run.trial080.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 20 kB 00:00 2026-02-19T20:06:10.994 INFO:teuthology.orchestra.run.trial080.stdout:Extra Packages for Enterprise Linux 39 MB/s | 20 MB 00:00 2026-02-19T20:06:13.498 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 9/10 2026-02-19T20:06:13.520 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 10/10 2026-02-19T20:06:13.874 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 10/10 2026-02-19T20:06:13.875 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 1/10 2026-02-19T20:06:13.875 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 2/10 2026-02-19T20:06:13.875 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/10 2026-02-19T20:06:13.875 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/10 2026-02-19T20:06:13.875 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/10 2026-02-19T20:06:13.875 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 6/10 2026-02-19T20:06:13.875 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 7/10 2026-02-19T20:06:13.875 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 8/10 2026-02-19T20:06:13.875 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 9/10 2026-02-19T20:06:13.930 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2026-02-19T20:06:13.930 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:13.930 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:13.931 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:13.969 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph 2026-02-19T20:06:14.144 INFO:teuthology.orchestra.run.trial080.stdout:lab-extras 1.8 MB/s | 50 kB 00:00 2026-02-19T20:06:14.269 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:22 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:14.354 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:14.357 INFO:teuthology.orchestra.run.trial073.stdout:======================================================================================= 2026-02-19T20:06:14.357 INFO:teuthology.orchestra.run.trial073.stdout: Package Arch Version Repository Size 2026-02-19T20:06:14.357 INFO:teuthology.orchestra.run.trial073.stdout:======================================================================================= 2026-02-19T20:06:14.357 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:06:14.357 INFO:teuthology.orchestra.run.trial073.stdout: ceph x86_64 2:20.3.0-5336.g802bf111.el9 ceph 6.5 k 2026-02-19T20:06:14.357 INFO:teuthology.orchestra.run.trial073.stdout:Installing dependencies: 2026-02-19T20:06:14.357 INFO:teuthology.orchestra.run.trial073.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-19T20:06:14.357 INFO:teuthology.orchestra.run.trial073.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 57 k 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mds x86_64 2:20.3.0-5336.g802bf111.el9 ceph 2.4 M 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr x86_64 2:20.3.0-5336.g802bf111.el9 ceph 935 k 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 297 k 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mon x86_64 2:20.3.0-5336.g802bf111.el9 ceph 5.0 M 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: ceph-osd x86_64 2:20.3.0-5336.g802bf111.el9 ceph 168 k 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: ceph-osd-classic x86_64 2:20.3.0-5336.g802bf111.el9 ceph 16 M 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: ceph-osd-crimson x86_64 2:20.3.0-5336.g802bf111.el9 ceph 19 M 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 17 k 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: cephadm noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 1.0 M 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-19T20:06:14.358 INFO:teuthology.orchestra.run.trial073.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-19T20:06:14.359 INFO:teuthology.orchestra.run.trial073.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-19T20:06:14.360 INFO:teuthology.orchestra.run.trial073.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-19T20:06:14.361 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-19T20:06:14.362 INFO:teuthology.orchestra.run.trial073.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-19T20:06:14.363 INFO:teuthology.orchestra.run.trial073.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-19T20:06:14.364 INFO:teuthology.orchestra.run.trial073.stdout:Installing weak dependencies: 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 184 k 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 51 M 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 7.4 M 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 22 k 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-rook noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 50 k 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout: ceph-volume noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 301 k 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:06:14.365 INFO:teuthology.orchestra.run.trial073.stdout:======================================================================================= 2026-02-19T20:06:14.366 INFO:teuthology.orchestra.run.trial073.stdout:Install 96 Packages 2026-02-19T20:06:14.366 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:14.366 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 159 M 2026-02-19T20:06:14.367 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 733 M 2026-02-19T20:06:14.367 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:06:14.951 INFO:teuthology.orchestra.run.trial073.stdout:(1/96): ceph-20.3.0-5336.g802bf111.el9.x86_64.r 120 kB/s | 6.5 kB 00:00 2026-02-19T20:06:15.015 INFO:teuthology.orchestra.run.trial073.stdout:(2/96): ceph-mgr-20.3.0-5336.g802bf111.el9.x86_ 7.7 MB/s | 935 kB 00:00 2026-02-19T20:06:15.034 INFO:teuthology.orchestra.run.trial073.stdout:(3/96): ceph-osd-20.3.0-5336.g802bf111.el9.x86_ 8.6 MB/s | 168 kB 00:00 2026-02-19T20:06:15.063 INFO:teuthology.orchestra.run.trial073.stdout:(4/96): ceph-mds-20.3.0-5336.g802bf111.el9.x86_ 14 MB/s | 2.4 MB 00:00 2026-02-19T20:06:15.274 INFO:teuthology.orchestra.run.trial073.stdout:(5/96): ceph-mon-20.3.0-5336.g802bf111.el9.x86_ 15 MB/s | 5.0 MB 00:00 2026-02-19T20:06:15.324 INFO:teuthology.orchestra.run.trial073.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5336.g80 1.1 MB/s | 57 kB 00:00 2026-02-19T20:06:15.336 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:15.337 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repository Size 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout: ceph-radosgw x86_64 2:20.3.0-5336.g802bf111.el9 ceph 13 M 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout:Upgrading: 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout: librados2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 3.6 M 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout: librbd1 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 2.8 M 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout:Installing dependencies: 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-19T20:06:15.338 INFO:teuthology.orchestra.run.trial080.stdout: ceph-base x86_64 2:20.3.0-5336.g802bf111.el9 ceph 5.2 M 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: ceph-common x86_64 2:20.3.0-5336.g802bf111.el9 ceph 24 M 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: ceph-selinux x86_64 2:20.3.0-5336.g802bf111.el9 ceph 25 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 910 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: libradosstriper1 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 250 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-19T20:06:15.339 INFO:teuthology.orchestra.run.trial080.stdout: librgw2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 6.5 M 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: python3-ceph-argparse x86_64 2:20.3.0-5336.g802bf111.el9 ceph 45 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: python3-ceph-common x86_64 2:20.3.0-5336.g802bf111.el9 ceph 185 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: python3-cephfs x86_64 2:20.3.0-5336.g802bf111.el9 ceph 203 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: python3-rados x86_64 2:20.3.0-5336.g802bf111.el9 ceph 323 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: python3-rbd x86_64 2:20.3.0-5336.g802bf111.el9 ceph 305 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: python3-rgw x86_64 2:20.3.0-5336.g802bf111.el9 ceph 100 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-19T20:06:15.340 INFO:teuthology.orchestra.run.trial080.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout:Installing weak dependencies: 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs-daemon x86_64 2:20.3.0-5336.g802bf111.el9 ceph 37 k 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5336.g802bf111.el9 ceph 25 k 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout:Install 35 Packages 2026-02-19T20:06:15.341 INFO:teuthology.orchestra.run.trial080.stdout:Upgrade 2 Packages 2026-02-19T20:06:15.342 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:15.342 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 68 M 2026-02-19T20:06:15.342 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:06:15.378 INFO:teuthology.orchestra.run.trial073.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5336.g802bf111. 3.3 MB/s | 184 kB 00:00 2026-02-19T20:06:15.871 INFO:teuthology.orchestra.run.trial073.stdout:(8/96): ceph-osd-crimson-20.3.0-5336.g802bf111. 24 MB/s | 19 MB 00:00 2026-02-19T20:06:15.893 INFO:teuthology.orchestra.run.trial073.stdout:(9/96): ceph-osd-classic-20.3.0-5336.g802bf111. 18 MB/s | 16 MB 00:00 2026-02-19T20:06:15.917 INFO:teuthology.orchestra.run.trial073.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5336.g802bf1 917 kB/s | 22 kB 00:00 2026-02-19T20:06:15.961 INFO:teuthology.orchestra.run.trial073.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5336.g802 6.7 MB/s | 297 kB 00:00 2026-02-19T20:06:16.000 INFO:teuthology.orchestra.run.trial073.stdout:(12/96): ceph-mgr-rook-20.3.0-5336.g802bf111.el 1.2 MB/s | 50 kB 00:00 2026-02-19T20:06:16.042 INFO:teuthology.orchestra.run.trial073.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5336.g80 421 kB/s | 17 kB 00:00 2026-02-19T20:06:16.089 INFO:teuthology.orchestra.run.trial073.stdout:(14/96): ceph-volume-20.3.0-5336.g802bf111.el9. 6.3 MB/s | 301 kB 00:00 2026-02-19T20:06:16.118 INFO:teuthology.orchestra.run.trial073.stdout:(15/96): ceph-mgr-diskprediction-local-20.3.0-5 30 MB/s | 7.4 MB 00:00 2026-02-19T20:06:16.151 INFO:teuthology.orchestra.run.trial073.stdout:(16/96): cephadm-20.3.0-5336.g802bf111.el9.noar 16 MB/s | 1.0 MB 00:00 2026-02-19T20:06:16.743 INFO:teuthology.orchestra.run.trial073.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 68 kB/s | 40 kB 00:00 2026-02-19T20:06:16.836 INFO:teuthology.orchestra.run.trial080.stdout:(1/37): ceph-base-20.3.0-5336.g802bf111.el9.x86 5.2 MB/s | 5.2 MB 00:00 2026-02-19T20:06:16.864 INFO:teuthology.orchestra.run.trial080.stdout:(2/37): ceph-selinux-20.3.0-5336.g802bf111.el9. 899 kB/s | 25 kB 00:00 2026-02-19T20:06:16.892 INFO:teuthology.orchestra.run.trial080.stdout:(3/37): libcephfs-daemon-20.3.0-5336.g802bf111. 1.3 MB/s | 37 kB 00:00 2026-02-19T20:06:16.922 INFO:teuthology.orchestra.run.trial080.stdout:(4/37): libcephfs-proxy2-20.3.0-5336.g802bf111. 871 kB/s | 25 kB 00:00 2026-02-19T20:06:16.978 INFO:teuthology.orchestra.run.trial073.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 307 kB/s | 72 kB 00:00 2026-02-19T20:06:17.028 INFO:teuthology.orchestra.run.trial080.stdout:(5/37): libcephfs2-20.3.0-5336.g802bf111.el9.x8 8.4 MB/s | 910 kB 00:00 2026-02-19T20:06:17.065 INFO:teuthology.orchestra.run.trial080.stdout:(6/37): libradosstriper1-20.3.0-5336.g802bf111. 6.7 MB/s | 250 kB 00:00 2026-02-19T20:06:17.078 INFO:teuthology.orchestra.run.trial073.stdout:(19/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 366 kB/s | 351 kB 00:00 2026-02-19T20:06:17.419 INFO:teuthology.orchestra.run.trial073.stdout:(20/96): ceph-mgr-dashboard-20.3.0-5336.g802bf1 25 MB/s | 51 MB 00:02 2026-02-19T20:06:17.420 INFO:teuthology.orchestra.run.trial073.stdout:(21/96): libquadmath-11.5.0-14.el9.x86_64.rpm 539 kB/s | 184 kB 00:00 2026-02-19T20:06:17.427 INFO:teuthology.orchestra.run.trial080.stdout:(7/37): ceph-radosgw-20.3.0-5336.g802bf111.el9. 8.4 MB/s | 13 MB 00:01 2026-02-19T20:06:17.451 INFO:teuthology.orchestra.run.trial080.stdout:(8/37): python3-ceph-argparse-20.3.0-5336.g802b 1.8 MB/s | 45 kB 00:00 2026-02-19T20:06:17.479 INFO:teuthology.orchestra.run.trial080.stdout:(9/37): python3-ceph-common-20.3.0-5336.g802bf1 6.6 MB/s | 185 kB 00:00 2026-02-19T20:06:17.508 INFO:teuthology.orchestra.run.trial080.stdout:(10/37): python3-cephfs-20.3.0-5336.g802bf111.e 6.7 MB/s | 203 kB 00:00 2026-02-19T20:06:17.532 INFO:teuthology.orchestra.run.trial080.stdout:(11/37): librgw2-20.3.0-5336.g802bf111.el9.x86_ 14 MB/s | 6.5 MB 00:00 2026-02-19T20:06:17.538 INFO:teuthology.orchestra.run.trial080.stdout:(12/37): python3-rados-20.3.0-5336.g802bf111.el 11 MB/s | 323 kB 00:00 2026-02-19T20:06:17.558 INFO:teuthology.orchestra.run.trial080.stdout:(13/37): python3-rbd-20.3.0-5336.g802bf111.el9. 11 MB/s | 305 kB 00:00 2026-02-19T20:06:17.561 INFO:teuthology.orchestra.run.trial080.stdout:(14/37): python3-rgw-20.3.0-5336.g802bf111.el9. 4.3 MB/s | 100 kB 00:00 2026-02-19T20:06:17.607 INFO:teuthology.orchestra.run.trial080.stdout:(15/37): mailcap-2.1.49-5.el9.noarch.rpm 683 kB/s | 33 kB 00:00 2026-02-19T20:06:17.642 INFO:teuthology.orchestra.run.trial080.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 2.6 MB/s | 93 kB 00:00 2026-02-19T20:06:17.650 INFO:teuthology.orchestra.run.trial080.stdout:(17/37): nvme-cli-2.16-1.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2026-02-19T20:06:17.711 INFO:teuthology.orchestra.run.trial080.stdout:(18/37): ceph-common-20.3.0-5336.g802bf111.el9. 13 MB/s | 24 MB 00:01 2026-02-19T20:06:17.821 INFO:teuthology.orchestra.run.trial073.stdout:(22/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 631 kB/s | 253 kB 00:00 2026-02-19T20:06:17.990 INFO:teuthology.orchestra.run.trial073.stdout:(23/96): libgfortran-11.5.0-14.el9.x86_64.rpm 785 kB/s | 794 kB 00:01 2026-02-19T20:06:18.061 INFO:teuthology.orchestra.run.trial073.stdout:(24/96): python3-ply-3.11-14.el9.noarch.rpm 443 kB/s | 106 kB 00:00 2026-02-19T20:06:18.230 INFO:teuthology.orchestra.run.trial073.stdout:(25/96): python3-pycparser-2.20-6.el9.noarch.rp 564 kB/s | 135 kB 00:00 2026-02-19T20:06:18.308 INFO:teuthology.orchestra.run.trial080.stdout:(19/37): boost-program-options-1.75.0-13.el9.x8 156 kB/s | 104 kB 00:00 2026-02-19T20:06:18.369 INFO:teuthology.orchestra.run.trial080.stdout:(20/37): libnbd-1.20.3-4.el9.x86_64.rpm 228 kB/s | 164 kB 00:00 2026-02-19T20:06:18.374 INFO:teuthology.orchestra.run.trial073.stdout:(26/96): python3-pyparsing-2.4.7-9.el9.noarch.r 481 kB/s | 150 kB 00:00 2026-02-19T20:06:18.430 INFO:teuthology.orchestra.run.trial080.stdout:(21/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 223 kB/s | 160 kB 00:00 2026-02-19T20:06:18.469 INFO:teuthology.orchestra.run.trial073.stdout:(27/96): python3-requests-2.25.1-10.el9.noarch. 528 kB/s | 126 kB 00:00 2026-02-19T20:06:18.765 INFO:teuthology.orchestra.run.trial073.stdout:(28/96): python3-urllib3-1.26.5-7.el9.noarch.rp 557 kB/s | 218 kB 00:00 2026-02-19T20:06:18.788 INFO:teuthology.orchestra.run.trial073.stdout:(29/96): unzip-6.0-59.el9.x86_64.rpm 570 kB/s | 182 kB 00:00 2026-02-19T20:06:18.835 INFO:teuthology.orchestra.run.trial073.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 637 kB/s | 30 kB 00:00 2026-02-19T20:06:18.888 INFO:teuthology.orchestra.run.trial080.stdout:(22/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2026-02-19T20:06:18.964 INFO:teuthology.orchestra.run.trial073.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 23 MB/s | 3.0 MB 00:00 2026-02-19T20:06:18.986 INFO:teuthology.orchestra.run.trial073.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 691 kB/s | 15 kB 00:00 2026-02-19T20:06:19.009 INFO:teuthology.orchestra.run.trial073.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 10 MB/s | 246 kB 00:00 2026-02-19T20:06:19.033 INFO:teuthology.orchestra.run.trial073.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 7.9 MB/s | 188 kB 00:00 2026-02-19T20:06:19.054 INFO:teuthology.orchestra.run.trial073.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-19T20:06:19.106 INFO:teuthology.orchestra.run.trial073.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 103 MB/s | 5.3 MB 00:00 2026-02-19T20:06:19.163 INFO:teuthology.orchestra.run.trial073.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 106 MB/s | 6.0 MB 00:00 2026-02-19T20:06:19.163 INFO:teuthology.orchestra.run.trial073.stdout:(38/96): zip-3.0-35.el9.x86_64.rpm 667 kB/s | 266 kB 00:00 2026-02-19T20:06:19.187 INFO:teuthology.orchestra.run.trial073.stdout:(39/96): python3-devel-3.9.25-3.el9.x86_64.rpm 9.7 MB/s | 244 kB 00:00 2026-02-19T20:06:19.210 INFO:teuthology.orchestra.run.trial073.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 2.1 MB/s | 48 kB 00:00 2026-02-19T20:06:19.213 INFO:teuthology.orchestra.run.trial080.stdout:(23/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 374 kB/s | 292 kB 00:00 2026-02-19T20:06:19.234 INFO:teuthology.orchestra.run.trial073.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 7.4 MB/s | 177 kB 00:00 2026-02-19T20:06:19.248 INFO:teuthology.orchestra.run.trial080.stdout:(24/37): librdkafka-1.6.1-102.el9.x86_64.rpm 754 kB/s | 662 kB 00:00 2026-02-19T20:06:19.249 INFO:teuthology.orchestra.run.trial073.stdout:(42/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 2.8 MB/s | 249 kB 00:00 2026-02-19T20:06:19.260 INFO:teuthology.orchestra.run.trial073.stdout:(43/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 45 MB/s | 1.2 MB 00:00 2026-02-19T20:06:19.272 INFO:teuthology.orchestra.run.trial073.stdout:(44/96): python3-markupsafe-1.1.1-12.el9.x86_64 1.5 MB/s | 35 kB 00:00 2026-02-19T20:06:19.300 INFO:teuthology.orchestra.run.trial073.stdout:(45/96): python3-cryptography-36.0.1-5.el9.x86_ 678 kB/s | 1.2 MB 00:01 2026-02-19T20:06:19.317 INFO:teuthology.orchestra.run.trial073.stdout:(46/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 109 MB/s | 6.1 MB 00:00 2026-02-19T20:06:19.321 INFO:teuthology.orchestra.run.trial073.stdout:(47/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 8.7 MB/s | 442 kB 00:00 2026-02-19T20:06:19.342 INFO:teuthology.orchestra.run.trial073.stdout:(48/96): python3-protobuf-3.14.0-17.el9.noarch. 11 MB/s | 267 kB 00:00 2026-02-19T20:06:19.345 INFO:teuthology.orchestra.run.trial073.stdout:(49/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 6.5 MB/s | 159 kB 00:00 2026-02-19T20:06:19.364 INFO:teuthology.orchestra.run.trial073.stdout:(50/96): python3-packaging-20.9-5.el9.noarch.rp 1.2 MB/s | 77 kB 00:00 2026-02-19T20:06:19.366 INFO:teuthology.orchestra.run.trial073.stdout:(51/96): python3-pyasn1-modules-0.4.8-6.el9.noa 11 MB/s | 279 kB 00:00 2026-02-19T20:06:19.368 INFO:teuthology.orchestra.run.trial073.stdout:(52/96): python3-requests-oauthlib-1.3.0-12.el9 2.3 MB/s | 54 kB 00:00 2026-02-19T20:06:19.389 INFO:teuthology.orchestra.run.trial073.stdout:(53/96): python3-toml-0.10.2-6.el9.noarch.rpm 1.8 MB/s | 42 kB 00:00 2026-02-19T20:06:19.392 INFO:teuthology.orchestra.run.trial073.stdout:(54/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 7.8 MB/s | 189 kB 00:00 2026-02-19T20:06:19.411 INFO:teuthology.orchestra.run.trial073.stdout:(55/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.9 MB/s | 90 kB 00:00 2026-02-19T20:06:19.413 INFO:teuthology.orchestra.run.trial073.stdout:(56/96): lua-devel-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 22 kB 00:00 2026-02-19T20:06:19.438 INFO:teuthology.orchestra.run.trial073.stdout:(57/96): protobuf-compiler-3.14.0-17.el9.x86_64 32 MB/s | 862 kB 00:00 2026-02-19T20:06:19.498 INFO:teuthology.orchestra.run.trial073.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 326 kB/s | 19 kB 00:00 2026-02-19T20:06:19.558 INFO:teuthology.orchestra.run.trial073.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2026-02-19T20:06:19.560 INFO:teuthology.orchestra.run.trial073.stdout:(60/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 3.7 MB/s | 551 kB 00:00 2026-02-19T20:06:19.600 INFO:teuthology.orchestra.run.trial073.stdout:(61/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 81 MB/s | 19 MB 00:00 2026-02-19T20:06:19.601 INFO:teuthology.orchestra.run.trial073.stdout:(62/96): python3-autocommand-2.2.2-8.el9.noarch 733 kB/s | 29 kB 00:00 2026-02-19T20:06:19.624 INFO:teuthology.orchestra.run.trial073.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.8 MB/s | 43 kB 00:00 2026-02-19T20:06:19.625 INFO:teuthology.orchestra.run.trial073.stdout:(64/96): python3-asyncssh-2.13.2-5.el9.noarch.r 8.0 MB/s | 548 kB 00:00 2026-02-19T20:06:19.645 INFO:teuthology.orchestra.run.trial073.stdout:(65/96): python3-cachetools-4.2.4-1.el9.noarch. 1.5 MB/s | 32 kB 00:00 2026-02-19T20:06:19.645 INFO:teuthology.orchestra.run.trial073.stdout:(66/96): python3-certifi-2023.05.07-4.el9.noarc 684 kB/s | 14 kB 00:00 2026-02-19T20:06:19.652 INFO:teuthology.orchestra.run.trial080.stdout:(25/37): qatlib-25.08.0-2.el9.x86_64.rpm 313 kB/s | 240 kB 00:00 2026-02-19T20:06:19.667 INFO:teuthology.orchestra.run.trial073.stdout:(67/96): python3-cheroot-10.0.1-4.el9.noarch.rp 7.5 MB/s | 173 kB 00:00 2026-02-19T20:06:19.669 INFO:teuthology.orchestra.run.trial073.stdout:(68/96): python3-cherrypy-18.6.1-2.el9.noarch.r 15 MB/s | 358 kB 00:00 2026-02-19T20:06:19.679 INFO:teuthology.orchestra.run.trial073.stdout:(69/96): python3-backports-tarfile-1.2.0-1.el9. 764 kB/s | 60 kB 00:00 2026-02-19T20:06:19.691 INFO:teuthology.orchestra.run.trial073.stdout:(70/96): python3-google-auth-2.45.0-1.el9.noarc 11 MB/s | 254 kB 00:00 2026-02-19T20:06:19.716 INFO:teuthology.orchestra.run.trial073.stdout:(71/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 43 MB/s | 2.0 MB 00:00 2026-02-19T20:06:19.716 INFO:teuthology.orchestra.run.trial073.stdout:(72/96): python3-influxdb-5.3.1-1.el9.noarch.rp 5.4 MB/s | 139 kB 00:00 2026-02-19T20:06:19.721 INFO:teuthology.orchestra.run.trial073.stdout:(73/96): python3-grpcio-tools-1.46.7-10.el9.x86 3.4 MB/s | 144 kB 00:00 2026-02-19T20:06:19.737 INFO:teuthology.orchestra.run.trial073.stdout:(74/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 513 kB/s | 11 kB 00:00 2026-02-19T20:06:19.738 INFO:teuthology.orchestra.run.trial073.stdout:(75/96): python3-isodate-0.6.1-3.el9.noarch.rpm 2.5 MB/s | 56 kB 00:00 2026-02-19T20:06:19.742 INFO:teuthology.orchestra.run.trial073.stdout:(76/96): python3-jaraco-classes-3.2.1-5.el9.noa 852 kB/s | 18 kB 00:00 2026-02-19T20:06:19.758 INFO:teuthology.orchestra.run.trial073.stdout:(77/96): python3-jaraco-collections-3.0.0-8.el9 1.1 MB/s | 23 kB 00:00 2026-02-19T20:06:19.759 INFO:teuthology.orchestra.run.trial073.stdout:(78/96): python3-jaraco-context-6.0.1-3.el9.noa 950 kB/s | 20 kB 00:00 2026-02-19T20:06:19.762 INFO:teuthology.orchestra.run.trial073.stdout:(79/96): python3-jaraco-functools-3.5.0-2.el9.n 941 kB/s | 19 kB 00:00 2026-02-19T20:06:19.779 INFO:teuthology.orchestra.run.trial073.stdout:(80/96): python3-jaraco-text-4.0.0-2.el9.noarch 1.2 MB/s | 26 kB 00:00 2026-02-19T20:06:19.782 INFO:teuthology.orchestra.run.trial080.stdout:(26/37): qatlib-service-25.08.0-2.el9.x86_64.rp 65 kB/s | 37 kB 00:00 2026-02-19T20:06:19.784 INFO:teuthology.orchestra.run.trial073.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 3.6 MB/s | 79 kB 00:00 2026-02-19T20:06:19.801 INFO:teuthology.orchestra.run.trial073.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 3.8 MB/s | 86 kB 00:00 2026-02-19T20:06:19.803 INFO:teuthology.orchestra.run.trial073.stdout:(83/96): python3-kubernetes-26.1.0-3.el9.noarch 23 MB/s | 1.0 MB 00:00 2026-02-19T20:06:19.805 INFO:teuthology.orchestra.run.trial073.stdout:(84/96): python3-natsort-7.1.1-5.el9.noarch.rpm 2.6 MB/s | 58 kB 00:00 2026-02-19T20:06:19.822 INFO:teuthology.orchestra.run.trial073.stdout:(85/96): python3-portend-3.1.0-2.el9.noarch.rpm 789 kB/s | 16 kB 00:00 2026-02-19T20:06:19.825 INFO:teuthology.orchestra.run.trial073.stdout:(86/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 4.0 MB/s | 90 kB 00:00 2026-02-19T20:06:19.826 INFO:teuthology.orchestra.run.trial073.stdout:(87/96): python3-repoze-lru-0.7-16.el9.noarch.r 1.4 MB/s | 31 kB 00:00 2026-02-19T20:06:19.845 INFO:teuthology.orchestra.run.trial073.stdout:(88/96): python3-routes-2.5.1-5.el9.noarch.rpm 8.1 MB/s | 188 kB 00:00 2026-02-19T20:06:19.846 INFO:teuthology.orchestra.run.trial073.stdout:(89/96): python3-rsa-4.9-2.el9.noarch.rpm 2.7 MB/s | 59 kB 00:00 2026-02-19T20:06:19.848 INFO:teuthology.orchestra.run.trial073.stdout:(90/96): python3-saml-1.16.0-1.el9.noarch.rpm 5.5 MB/s | 125 kB 00:00 2026-02-19T20:06:19.851 INFO:teuthology.orchestra.run.trial080.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.5 MB/s | 308 kB 00:00 2026-02-19T20:06:19.866 INFO:teuthology.orchestra.run.trial073.stdout:(91/96): python3-tempora-5.0.0-2.el9.noarch.rpm 1.7 MB/s | 36 kB 00:00 2026-02-19T20:06:19.868 INFO:teuthology.orchestra.run.trial073.stdout:(92/96): python3-typing-extensions-4.15.0-1.el9 3.9 MB/s | 86 kB 00:00 2026-02-19T20:06:19.870 INFO:teuthology.orchestra.run.trial073.stdout:(93/96): python3-websocket-client-1.2.3-2.el9.n 4.0 MB/s | 90 kB 00:00 2026-02-19T20:06:19.872 INFO:teuthology.orchestra.run.trial080.stdout:(28/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 106 kB/s | 66 kB 00:00 2026-02-19T20:06:19.888 INFO:teuthology.orchestra.run.trial073.stdout:(94/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 2.2 MB/s | 48 kB 00:00 2026-02-19T20:06:19.890 INFO:teuthology.orchestra.run.trial073.stdout:(95/96): python3-xmltodict-0.12.0-15.el9.noarch 1.0 MB/s | 22 kB 00:00 2026-02-19T20:06:19.891 INFO:teuthology.orchestra.run.trial073.stdout:(96/96): python3-zc-lockfile-2.0-10.el9.noarch. 957 kB/s | 20 kB 00:00 2026-02-19T20:06:19.893 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:19.893 INFO:teuthology.orchestra.run.trial073.stdout:Total 29 MB/s | 159 MB 00:05 2026-02-19T20:06:19.925 INFO:teuthology.orchestra.run.trial080.stdout:(29/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 335 kB/s | 25 kB 00:00 2026-02-19T20:06:19.951 INFO:teuthology.orchestra.run.trial080.stdout:(30/37): libarrow-9.0.0-15.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2026-02-19T20:06:19.952 INFO:teuthology.orchestra.run.trial080.stdout:(31/37): liboath-2.6.12-1.el9.x86_64.rpm 616 kB/s | 49 kB 00:00 2026-02-19T20:06:19.952 INFO:teuthology.orchestra.run.trial080.stdout:(32/37): libunwind-1.6.2-1.el9.x86_64.rpm 2.4 MB/s | 67 kB 00:00 2026-02-19T20:06:19.980 INFO:teuthology.orchestra.run.trial080.stdout:(33/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 29 MB/s | 838 kB 00:00 2026-02-19T20:06:19.980 INFO:teuthology.orchestra.run.trial080.stdout:(34/37): re2-20211101-20.el9.x86_64.rpm 6.5 MB/s | 191 kB 00:00 2026-02-19T20:06:19.987 INFO:teuthology.orchestra.run.trial080.stdout:(35/37): thrift-0.15.0-4.el9.x86_64.rpm 46 MB/s | 1.6 MB 00:00 2026-02-19T20:06:20.174 INFO:teuthology.orchestra.run.trial080.stdout:(36/37): librbd1-20.3.0-5336.g802bf111.el9.x86_ 15 MB/s | 2.8 MB 00:00 2026-02-19T20:06:20.214 INFO:teuthology.orchestra.run.trial080.stdout:(37/37): librados2-20.3.0-5336.g802bf111.el9.x8 15 MB/s | 3.6 MB 00:00 2026-02-19T20:06:20.217 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:20.218 INFO:teuthology.orchestra.run.trial080.stdout:Total 14 MB/s | 68 MB 00:04 2026-02-19T20:06:20.314 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:06:20.319 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:06:20.338 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:06:20.338 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:06:20.352 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:06:20.352 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:06:20.610 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:06:20.610 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:06:20.932 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:06:20.952 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:06:20.952 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:06:21.089 INFO:teuthology.orchestra.run.trial080.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-19T20:06:21.094 INFO:teuthology.orchestra.run.trial080.stdout: Upgrading : librados2-2:20.3.0-5336.g802bf111.el9.x86_64 2/39 2026-02-19T20:06:21.145 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librados2-2:20.3.0-5336.g802bf111.el9.x86_64 2/39 2026-02-19T20:06:21.146 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 3/39 2026-02-19T20:06:21.159 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 3/39 2026-02-19T20:06:21.161 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libcephfs-daemon-2:20.3.0-5336.g802bf111.el9.x86_6 4/39 2026-02-19T20:06:21.162 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_6 5/39 2026-02-19T20:06:21.190 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_6 5/39 2026-02-19T20:06:21.194 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 6/39 2026-02-19T20:06:21.203 INFO:teuthology.orchestra.run.trial080.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-19T20:06:21.206 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-19T20:06:21.209 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-19T20:06:21.213 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-19T20:06:21.224 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-ceph-argparse-2:20.3.0-5336.g802bf111.el9. 11/39 2026-02-19T20:06:21.236 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 12/39 2026-02-19T20:06:21.236 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_6 13/39 2026-02-19T20:06:21.258 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_6 13/39 2026-02-19T20:06:21.262 INFO:teuthology.orchestra.run.trial080.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-19T20:06:21.292 INFO:teuthology.orchestra.run.trial080.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-19T20:06:21.296 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-19T20:06:21.302 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-19T20:06:21.305 INFO:teuthology.orchestra.run.trial080.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-19T20:06:21.310 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-19T20:06:21.432 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-19T20:06:21.433 INFO:teuthology.orchestra.run.trial080.stdout: Upgrading : librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 21/39 2026-02-19T20:06:21.462 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 21/39 2026-02-19T20:06:21.465 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 22/39 2026-02-19T20:06:21.472 INFO:teuthology.orchestra.run.trial080.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-19T20:06:21.594 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:06:21.596 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-19T20:06:21.600 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-19T20:06:21.621 INFO:teuthology.orchestra.run.trial073.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-19T20:06:21.654 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-19T20:06:21.661 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-19T20:06:21.663 INFO:teuthology.orchestra.run.trial073.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-19T20:06:21.667 INFO:teuthology.orchestra.run.trial073.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-19T20:06:21.690 INFO:teuthology.orchestra.run.trial073.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-19T20:06:21.703 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-19T20:06:21.706 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-19T20:06:21.712 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-19T20:06:21.714 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-19T20:06:21.725 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-19T20:06:21.731 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-19T20:06:21.754 INFO:teuthology.orchestra.run.trial080.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-19T20:06:21.755 INFO:teuthology.orchestra.run.trial073.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-19T20:06:21.755 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 25/39 2026-02-19T20:06:21.758 INFO:teuthology.orchestra.run.trial073.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-19T20:06:21.773 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 25/39 2026-02-19T20:06:21.778 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 26/39 2026-02-19T20:06:21.784 INFO:teuthology.orchestra.run.trial080.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-19T20:06:21.786 INFO:teuthology.orchestra.run.trial080.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-19T20:06:21.787 INFO:teuthology.orchestra.run.trial080.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-19T20:06:21.806 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-19T20:06:21.808 INFO:teuthology.orchestra.run.trial073.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-19T20:06:21.821 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-19T20:06:21.823 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-19T20:06:21.828 INFO:teuthology.orchestra.run.trial073.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-19T20:06:21.833 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-19T20:06:21.838 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-19T20:06:21.841 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-19T20:06:21.855 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-19T20:06:21.860 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-19T20:06:21.865 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-19T20:06:21.876 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-19T20:06:21.885 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-19T20:06:21.891 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-19T20:06:21.910 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-19T20:06:21.922 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-19T20:06:21.924 INFO:teuthology.orchestra.run.trial080.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-19T20:06:21.926 INFO:teuthology.orchestra.run.trial080.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-19T20:06:21.937 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-19T20:06:21.941 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-19T20:06:21.949 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-19T20:06:21.989 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-19T20:06:22.206 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-19T20:06:22.206 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-19T20:06:22.206 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:22.251 INFO:teuthology.orchestra.run.trial080.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-19T20:06:22.253 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-ceph-common-2:20.3.0-5336.g802bf111.el9.x8 33/39 2026-02-19T20:06:22.313 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-19T20:06:22.323 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-19T20:06:22.327 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-19T20:06:22.333 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-19T20:06:22.335 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-19T20:06:22.362 INFO:teuthology.orchestra.run.trial073.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-19T20:06:22.406 INFO:teuthology.orchestra.run.trial073.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-19T20:06:22.414 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-19T20:06:22.420 INFO:teuthology.orchestra.run.trial073.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-19T20:06:22.424 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-19T20:06:22.429 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-19T20:06:22.433 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-19T20:06:22.440 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-19T20:06:22.443 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-19T20:06:22.471 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-19T20:06:22.482 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-19T20:06:22.488 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-19T20:06:22.494 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-19T20:06:22.529 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-19T20:06:22.752 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-19T20:06:22.776 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-19T20:06:22.780 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-19T20:06:22.838 INFO:teuthology.orchestra.run.trial073.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-19T20:06:22.839 INFO:teuthology.orchestra.run.trial073.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-19T20:06:22.861 INFO:teuthology.orchestra.run.trial073.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-19T20:06:23.187 INFO:teuthology.orchestra.run.trial073.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-19T20:06:23.259 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-19T20:06:23.344 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 34/39 2026-02-19T20:06:23.347 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 34/39 2026-02-19T20:06:23.611 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 34/39 2026-02-19T20:06:23.615 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 35/39 2026-02-19T20:06:23.655 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 35/39 2026-02-19T20:06:23.655 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-19T20:06:23.655 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-19T20:06:23.655 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:23.658 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 36/39 2026-02-19T20:06:23.933 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-19T20:06:23.949 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-19T20:06:23.959 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-19T20:06:23.966 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-19T20:06:23.979 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-19T20:06:23.994 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-19T20:06:24.070 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-19T20:06:24.078 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-19T20:06:24.101 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-19T20:06:24.131 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-19T20:06:24.181 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-19T20:06:24.187 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-19T20:06:24.190 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-19T20:06:24.192 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-19T20:06:24.210 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-19T20:06:24.210 INFO:teuthology.orchestra.run.trial073.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-19T20:06:24.210 INFO:teuthology.orchestra.run.trial073.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-19T20:06:24.210 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:24.217 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-19T20:06:24.243 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-19T20:06:24.243 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-19T20:06:24.243 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:24.278 INFO:teuthology.orchestra.run.trial073.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-19T20:06:24.899 INFO:teuthology.orchestra.run.trial073.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-19T20:06:25.667 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_6 80/96 2026-02-19T20:06:25.730 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_6 81/96 2026-02-19T20:06:25.732 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 82/96 2026-02-19T20:06:25.747 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 82/96 2026-02-19T20:06:25.747 INFO:teuthology.orchestra.run.trial073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:25.747 INFO:teuthology.orchestra.run.trial073.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-19T20:06:25.747 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:25.757 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 83/96 2026-02-19T20:06:25.779 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 83/96 2026-02-19T20:06:25.779 INFO:teuthology.orchestra.run.trial073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:25.779 INFO:teuthology.orchestra.run.trial073.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-19T20:06:25.779 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-19T20:06:25.779 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-19T20:06:25.779 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:25.841 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: cephadm-2:20.3.0-5336.g802bf111.el9.noarch 84/96 2026-02-19T20:06:25.843 INFO:teuthology.orchestra.run.trial073.stdout: Installing : cephadm-2:20.3.0-5336.g802bf111.el9.noarch 84/96 2026-02-19T20:06:25.852 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5336.g802bf111.el9 85/96 2026-02-19T20:06:25.875 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5336.g802bf111.el 86/96 2026-02-19T20:06:25.876 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarc 87/96 2026-02-19T20:06:28.181 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 36/39 2026-02-19T20:06:28.181 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /sys 2026-02-19T20:06:28.181 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /proc 2026-02-19T20:06:28.181 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /mnt 2026-02-19T20:06:28.181 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /var/tmp 2026-02-19T20:06:28.182 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /home 2026-02-19T20:06:28.182 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /root 2026-02-19T20:06:28.182 INFO:teuthology.orchestra.run.trial080.stdout:skipping the directory /tmp 2026-02-19T20:06:28.182 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:28.757 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 37/39 2026-02-19T20:06:28.765 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarc 87/96 2026-02-19T20:06:28.772 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noa 88/96 2026-02-19T20:06:28.780 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 37/39 2026-02-19T20:06:28.781 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:28.781 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-19T20:06:28.781 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-19T20:06:28.781 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-19T20:06:28.781 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:28.782 INFO:teuthology.orchestra.run.trial080.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-19T20:06:28.794 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-19T20:06:28.794 INFO:teuthology.orchestra.run.trial080.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-19T20:06:28.852 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-19T20:06:28.852 INFO:teuthology.orchestra.run.trial080.stdout:Creating group 'qat' with GID 992. 2026-02-19T20:06:28.852 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:29.155 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 1/39 2026-02-19T20:06:29.156 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 2/39 2026-02-19T20:06:29.156 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 3/39 2026-02-19T20:06:29.156 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 4/39 2026-02-19T20:06:29.156 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libcephfs-daemon-2:20.3.0-5336.g802bf111.el9.x86_6 5/39 2026-02-19T20:06:29.156 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_6 6/39 2026-02-19T20:06:29.156 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 7/39 2026-02-19T20:06:29.156 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_6 8/39 2026-02-19T20:06:29.156 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 9/39 2026-02-19T20:06:29.156 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5336.g802bf111.el9. 10/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-ceph-common-2:20.3.0-5336.g802bf111.el9.x8 11/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 12/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 13/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 14/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 15/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-19T20:06:29.157 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-19T20:06:29.158 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-19T20:06:29.159 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librados2-2:20.3.0-5336.g802bf111.el9.x86_64 36/39 2026-02-19T20:06:29.159 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-19T20:06:29.159 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 38/39 2026-02-19T20:06:29.216 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-19T20:06:29.217 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:29.217 INFO:teuthology.orchestra.run.trial080.stdout:Upgraded: 2026-02-19T20:06:29.217 INFO:teuthology.orchestra.run.trial080.stdout: librados2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.217 INFO:teuthology.orchestra.run.trial080.stdout: librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.217 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:06:29.217 INFO:teuthology.orchestra.run.trial080.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-19T20:06:29.217 INFO:teuthology.orchestra.run.trial080.stdout: ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.218 INFO:teuthology.orchestra.run.trial080.stdout: ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.218 INFO:teuthology.orchestra.run.trial080.stdout: ceph-radosgw-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.218 INFO:teuthology.orchestra.run.trial080.stdout: ceph-selinux-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.218 INFO:teuthology.orchestra.run.trial080.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-19T20:06:29.218 INFO:teuthology.orchestra.run.trial080.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-19T20:06:29.218 INFO:teuthology.orchestra.run.trial080.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-19T20:06:29.218 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs-daemon-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.219 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs-proxy2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.219 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.219 INFO:teuthology.orchestra.run.trial080.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-19T20:06:29.219 INFO:teuthology.orchestra.run.trial080.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-19T20:06:29.219 INFO:teuthology.orchestra.run.trial080.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-19T20:06:29.219 INFO:teuthology.orchestra.run.trial080.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-19T20:06:29.219 INFO:teuthology.orchestra.run.trial080.stdout: libradosstriper1-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.219 INFO:teuthology.orchestra.run.trial080.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-19T20:06:29.220 INFO:teuthology.orchestra.run.trial080.stdout: librgw2-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.220 INFO:teuthology.orchestra.run.trial080.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-19T20:06:29.220 INFO:teuthology.orchestra.run.trial080.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-19T20:06:29.220 INFO:teuthology.orchestra.run.trial080.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-19T20:06:29.220 INFO:teuthology.orchestra.run.trial080.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-19T20:06:29.220 INFO:teuthology.orchestra.run.trial080.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-19T20:06:29.220 INFO:teuthology.orchestra.run.trial080.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-19T20:06:29.220 INFO:teuthology.orchestra.run.trial080.stdout: python3-ceph-argparse-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.221 INFO:teuthology.orchestra.run.trial080.stdout: python3-ceph-common-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.221 INFO:teuthology.orchestra.run.trial080.stdout: python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.221 INFO:teuthology.orchestra.run.trial080.stdout: python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.221 INFO:teuthology.orchestra.run.trial080.stdout: python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.221 INFO:teuthology.orchestra.run.trial080.stdout: python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:29.221 INFO:teuthology.orchestra.run.trial080.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-19T20:06:29.221 INFO:teuthology.orchestra.run.trial080.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-19T20:06:29.221 INFO:teuthology.orchestra.run.trial080.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-19T20:06:29.222 INFO:teuthology.orchestra.run.trial080.stdout: re2-1:20211101-20.el9.x86_64 2026-02-19T20:06:29.222 INFO:teuthology.orchestra.run.trial080.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-19T20:06:29.222 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:29.222 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:29.225 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noa 88/96 2026-02-19T20:06:29.226 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf 89/96 2026-02-19T20:06:29.239 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf 89/96 2026-02-19T20:06:29.239 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noa 90/96 2026-02-19T20:06:29.259 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-test 2026-02-19T20:06:29.301 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noa 90/96 2026-02-19T20:06:29.342 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5336.g802bf111.el9. 91/96 2026-02-19T20:06:29.343 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 92/96 2026-02-19T20:06:29.365 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 92/96 2026-02-19T20:06:29.365 INFO:teuthology.orchestra.run.trial073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:29.365 INFO:teuthology.orchestra.run.trial073.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-19T20:06:29.365 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-19T20:06:29.365 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-19T20:06:29.365 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:29.377 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 93/96 2026-02-19T20:06:29.567 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:15 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:29.602 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 93/96 2026-02-19T20:06:29.603 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 94/96 2026-02-19T20:06:29.624 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 94/96 2026-02-19T20:06:29.624 INFO:teuthology.orchestra.run.trial073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:29.624 INFO:teuthology.orchestra.run.trial073.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-19T20:06:29.624 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-19T20:06:29.624 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-19T20:06:29.624 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:29.649 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repository Size 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: ceph-test x86_64 2:20.3.0-5336.g802bf111.el9 ceph 79 M 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:Installing dependencies: 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: libcephsqlite x86_64 2:20.3.0-5336.g802bf111.el9 ceph 159 k 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:Install 10 Packages 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 83 M 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 341 M 2026-02-19T20:06:29.650 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:06:29.722 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 95/96 2026-02-19T20:06:29.741 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 95/96 2026-02-19T20:06:29.741 INFO:teuthology.orchestra.run.trial073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:29.741 INFO:teuthology.orchestra.run.trial073.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-19T20:06:29.741 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-19T20:06:29.741 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-19T20:06:29.741 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:29.742 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-2:20.3.0-5336.g802bf111.el9.x86_64 96/96 2026-02-19T20:06:29.749 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_6 96/96 2026-02-19T20:06:29.774 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_6 96/96 2026-02-19T20:06:30.186 INFO:teuthology.orchestra.run.trial080.stdout:(1/10): libcephsqlite-20.3.0-5336.g802bf111.el9 1.5 MB/s | 159 kB 00:00 2026-02-19T20:06:30.433 INFO:teuthology.orchestra.run.trial080.stdout:(2/10): libatomic-11.5.0-14.el9.x86_64.rpm 92 kB/s | 23 kB 00:00 2026-02-19T20:06:30.604 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-2:20.3.0-5336.g802bf111.el9.x86_64 96/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-2:20.3.0-5336.g802bf111.el9.x86_64 1/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 2/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 3/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 4/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 5/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_6 6/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_6 7/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5336.g802bf111.el 8/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarc 9/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noa 10/96 2026-02-19T20:06:30.605 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf 11/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noa 12/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5336.g802bf111.el9. 13/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 14/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5336.g802bf111.el9 15/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 16/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : cephadm-2:20.3.0-5336.g802bf111.el9.noarch 17/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-19T20:06:30.606 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-19T20:06:30.607 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-19T20:06:30.608 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-19T20:06:30.609 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-19T20:06:30.610 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-19T20:06:30.611 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-19T20:06:30.612 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-19T20:06:30.612 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-19T20:06:30.612 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-19T20:06:30.612 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-19T20:06:30.612 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-19T20:06:30.612 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-19T20:06:30.612 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-19T20:06:30.667 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-19T20:06:30.668 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:30.668 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:30.668 INFO:teuthology.orchestra.run.trial073.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-19T20:06:30.668 INFO:teuthology.orchestra.run.trial073.stdout: ceph-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:30.668 INFO:teuthology.orchestra.run.trial073.stdout: ceph-grafana-dashboards-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.668 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-modules-core-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-prometheus-alerts-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.669 INFO:teuthology.orchestra.run.trial073.stdout: ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: cephadm-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-19T20:06:30.670 INFO:teuthology.orchestra.run.trial073.stdout: lua-5.4.4-4.el9.x86_64 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-19T20:06:30.671 INFO:teuthology.orchestra.run.trial073.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-19T20:06:30.672 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-19T20:06:30.673 INFO:teuthology.orchestra.run.trial073.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-ply-3.11-14.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-19T20:06:30.674 INFO:teuthology.orchestra.run.trial073.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-19T20:06:30.675 INFO:teuthology.orchestra.run.trial073.stdout: unzip-6.0-59.el9.x86_64 2026-02-19T20:06:30.676 INFO:teuthology.orchestra.run.trial073.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-19T20:06:30.676 INFO:teuthology.orchestra.run.trial073.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-19T20:06:30.676 INFO:teuthology.orchestra.run.trial073.stdout: zip-3.0-35.el9.x86_64 2026-02-19T20:06:30.676 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:30.676 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:30.676 INFO:teuthology.orchestra.run.trial080.stdout:(3/10): lksctp-tools-1.0.19-2.el9.x86_64.rpm 397 kB/s | 94 kB 00:00 2026-02-19T20:06:30.730 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-base 2026-02-19T20:06:30.755 INFO:teuthology.orchestra.run.trial080.stdout:(4/10): libxslt-1.1.34-12.el9.x86_64.rpm 2.7 MB/s | 233 kB 00:00 2026-02-19T20:06:30.797 INFO:teuthology.orchestra.run.trial080.stdout:(5/10): protobuf-3.14.0-17.el9.x86_64.rpm 24 MB/s | 1.0 MB 00:00 2026-02-19T20:06:30.825 INFO:teuthology.orchestra.run.trial080.stdout:(6/10): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.8 MB/s | 2.1 MB 00:00 2026-02-19T20:06:30.834 INFO:teuthology.orchestra.run.trial080.stdout:(7/10): socat-1.7.4.1-8.el9.x86_64.rpm 7.9 MB/s | 303 kB 00:00 2026-02-19T20:06:30.857 INFO:teuthology.orchestra.run.trial080.stdout:(8/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.0 MB/s | 64 kB 00:00 2026-02-19T20:06:31.036 INFO:teuthology.orchestra.run.trial080.stdout:(9/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 605 kB/s | 122 kB 00:00 2026-02-19T20:06:31.039 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:39 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:31.104 INFO:teuthology.orchestra.run.trial073.stdout:Package ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:31.122 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:31.123 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:31.123 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:31.144 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install cephadm 2026-02-19T20:06:31.449 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:39 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:31.514 INFO:teuthology.orchestra.run.trial073.stdout:Package cephadm-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:31.532 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:31.532 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:31.532 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:31.554 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-immutable-object-cache 2026-02-19T20:06:31.747 INFO:teuthology.orchestra.run.trial080.stdout:(10/10): ceph-test-20.3.0-5336.g802bf111.el9.x8 47 MB/s | 79 MB 00:01 2026-02-19T20:06:31.749 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:31.749 INFO:teuthology.orchestra.run.trial080.stdout:Total 39 MB/s | 83 MB 00:02 2026-02-19T20:06:31.814 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:06:31.824 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:06:31.824 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:06:31.861 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:39 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:31.944 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:31.945 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:31.945 INFO:teuthology.orchestra.run.trial073.stdout: Package Arch Version Repo Size 2026-02-19T20:06:31.945 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:31.945 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:06:31.946 INFO:teuthology.orchestra.run.trial073.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5336.g802bf111.el9 ceph 147 k 2026-02-19T20:06:31.946 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:31.946 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:06:31.946 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:31.946 INFO:teuthology.orchestra.run.trial073.stdout:Install 1 Package 2026-02-19T20:06:31.946 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:31.946 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 147 k 2026-02-19T20:06:31.946 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 423 k 2026-02-19T20:06:31.947 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:06:32.032 INFO:teuthology.orchestra.run.trial073.stdout:ceph-immutable-object-cache-20.3.0-5336.g802bf1 1.7 MB/s | 147 kB 00:00 2026-02-19T20:06:32.032 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:32.033 INFO:teuthology.orchestra.run.trial073.stdout:Total 1.6 MB/s | 147 kB 00:00 2026-02-19T20:06:32.033 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:06:32.037 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:06:32.037 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:06:32.068 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:06:32.068 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:06:32.109 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:06:32.109 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:06:32.123 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:06:32.125 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5336.g802bf111. 1/1 2026-02-19T20:06:32.148 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5336.g802bf111. 1/1 2026-02-19T20:06:32.148 INFO:teuthology.orchestra.run.trial073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:32.148 INFO:teuthology.orchestra.run.trial073.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-19T20:06:32.148 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:32.425 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:06:32.434 INFO:teuthology.orchestra.run.trial080.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2026-02-19T20:06:32.443 INFO:teuthology.orchestra.run.trial080.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/10 2026-02-19T20:06:32.447 INFO:teuthology.orchestra.run.trial080.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/10 2026-02-19T20:06:32.451 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/10 2026-02-19T20:06:32.457 INFO:teuthology.orchestra.run.trial080.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2026-02-19T20:06:32.460 INFO:teuthology.orchestra.run.trial080.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 6/10 2026-02-19T20:06:32.462 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5336.g802bf111. 1/1 2026-02-19T20:06:32.462 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:32.462 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:32.463 INFO:teuthology.orchestra.run.trial073.stdout: ceph-immutable-object-cache-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:32.463 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:32.463 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:32.470 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 7/10 2026-02-19T20:06:32.478 INFO:teuthology.orchestra.run.trial080.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 8/10 2026-02-19T20:06:32.479 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 9/10 2026-02-19T20:06:32.490 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-mgr 2026-02-19T20:06:32.797 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:40 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:32.862 INFO:teuthology.orchestra.run.trial073.stdout:Package ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:32.880 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:32.881 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:32.881 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:32.903 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-mgr-dashboard 2026-02-19T20:06:33.204 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:41 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:33.268 INFO:teuthology.orchestra.run.trial073.stdout:Package ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:33.286 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:33.287 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:33.287 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:33.308 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-19T20:06:33.616 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:41 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:33.681 INFO:teuthology.orchestra.run.trial073.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:33.699 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:33.700 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:33.700 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:33.721 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-mgr-rook 2026-02-19T20:06:34.029 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:42 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:34.093 INFO:teuthology.orchestra.run.trial073.stdout:Package ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:34.112 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:34.112 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:34.112 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:34.133 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-mgr-cephadm 2026-02-19T20:06:34.438 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:42 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:34.503 INFO:teuthology.orchestra.run.trial073.stdout:Package ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:34.521 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:34.522 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:34.522 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:34.543 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-osd 2026-02-19T20:06:34.846 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:42 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:34.911 INFO:teuthology.orchestra.run.trial073.stdout:Package ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:34.929 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:34.930 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:34.930 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:34.950 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-osd-classic 2026-02-19T20:06:35.254 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:43 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:35.321 INFO:teuthology.orchestra.run.trial073.stdout:Package ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:35.339 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:35.340 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:35.340 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:35.362 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-fuse 2026-02-19T20:06:35.667 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:43 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:35.750 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout: Package Arch Version Repo Size 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout: ceph-fuse x86_64 2:20.3.0-5336.g802bf111.el9 ceph 987 k 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout:Install 1 Package 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 987 k 2026-02-19T20:06:35.751 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 2.9 M 2026-02-19T20:06:35.752 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:06:35.871 INFO:teuthology.orchestra.run.trial073.stdout:ceph-fuse-20.3.0-5336.g802bf111.el9.x86_64.rpm 8.0 MB/s | 987 kB 00:00 2026-02-19T20:06:35.872 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:35.872 INFO:teuthology.orchestra.run.trial073.stdout:Total 7.9 MB/s | 987 kB 00:00 2026-02-19T20:06:35.872 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:06:35.877 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:06:35.877 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:06:35.912 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:06:35.912 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:06:36.000 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 9/10 2026-02-19T20:06:36.004 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:06:36.018 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 10/10 2026-02-19T20:06:36.021 INFO:teuthology.orchestra.run.trial073.stdout: Installing : ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:36.252 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:36.310 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:36.310 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:36.310 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:36.310 INFO:teuthology.orchestra.run.trial073.stdout: ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:36.310 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:36.310 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:36.341 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install ceph-volume 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 10/10 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 1/10 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 2/10 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/10 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/10 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/10 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 6/10 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 7/10 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 8/10 2026-02-19T20:06:36.369 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 9/10 2026-02-19T20:06:36.423 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2026-02-19T20:06:36.423 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:36.423 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:06:36.423 INFO:teuthology.orchestra.run.trial080.stdout: ceph-test-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:36.423 INFO:teuthology.orchestra.run.trial080.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout: libcephsqlite-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:36.424 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:36.462 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph 2026-02-19T20:06:36.651 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:44 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:36.715 INFO:teuthology.orchestra.run.trial073.stdout:Package ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:36.733 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:36.733 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:36.733 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:36.755 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install librados-devel 2026-02-19T20:06:36.768 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:22 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:36.856 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout:======================================================================================= 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repository Size 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout:======================================================================================= 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph x86_64 2:20.3.0-5336.g802bf111.el9 ceph 6.5 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout:Installing dependencies: 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 57 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mds x86_64 2:20.3.0-5336.g802bf111.el9 ceph 2.4 M 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr x86_64 2:20.3.0-5336.g802bf111.el9 ceph 935 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 297 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mon x86_64 2:20.3.0-5336.g802bf111.el9 ceph 5.0 M 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd x86_64 2:20.3.0-5336.g802bf111.el9 ceph 168 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd-classic x86_64 2:20.3.0-5336.g802bf111.el9 ceph 16 M 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd-crimson x86_64 2:20.3.0-5336.g802bf111.el9 ceph 19 M 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 17 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: cephadm noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 1.0 M 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-19T20:06:36.859 INFO:teuthology.orchestra.run.trial080.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-19T20:06:36.860 INFO:teuthology.orchestra.run.trial080.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout:Installing weak dependencies: 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 184 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 51 M 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 7.4 M 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 22 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-rook noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 50 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: ceph-volume noarch 2:20.3.0-5336.g802bf111.el9 ceph-noarch 301 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout:======================================================================================= 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout:Install 96 Packages 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 159 M 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 733 M 2026-02-19T20:06:36.861 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:06:37.058 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:45 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:37.141 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:37.141 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:37.141 INFO:teuthology.orchestra.run.trial073.stdout: Package Arch Version Repo Size 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout: librados-devel x86_64 2:20.3.0-5336.g802bf111.el9 ceph 123 k 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout:Install 1 Package 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 123 k 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 437 k 2026-02-19T20:06:37.142 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:06:37.227 INFO:teuthology.orchestra.run.trial073.stdout:librados-devel-20.3.0-5336.g802bf111.el9.x86_64 1.4 MB/s | 123 kB 00:00 2026-02-19T20:06:37.228 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:37.228 INFO:teuthology.orchestra.run.trial073.stdout:Total 1.4 MB/s | 123 kB 00:00 2026-02-19T20:06:37.228 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:06:37.230 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:06:37.231 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:06:37.262 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:06:37.262 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:06:37.316 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:06:37.333 INFO:teuthology.orchestra.run.trial073.stdout: Installing : librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:37.413 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:37.445 INFO:teuthology.orchestra.run.trial080.stdout:(1/96): ceph-20.3.0-5336.g802bf111.el9.x86_64.r 125 kB/s | 6.5 kB 00:00 2026-02-19T20:06:37.470 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:37.470 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:37.470 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:37.471 INFO:teuthology.orchestra.run.trial073.stdout: librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:37.471 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:37.471 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:37.498 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install libcephfs2 2026-02-19T20:06:37.535 INFO:teuthology.orchestra.run.trial080.stdout:(2/96): ceph-mgr-20.3.0-5336.g802bf111.el9.x86_ 6.4 MB/s | 935 kB 00:00 2026-02-19T20:06:37.565 INFO:teuthology.orchestra.run.trial080.stdout:(3/96): ceph-mds-20.3.0-5336.g802bf111.el9.x86_ 14 MB/s | 2.4 MB 00:00 2026-02-19T20:06:37.565 INFO:teuthology.orchestra.run.trial080.stdout:(4/96): ceph-osd-20.3.0-5336.g802bf111.el9.x86_ 5.4 MB/s | 168 kB 00:00 2026-02-19T20:06:37.734 INFO:teuthology.orchestra.run.trial080.stdout:(5/96): ceph-mon-20.3.0-5336.g802bf111.el9.x86_ 17 MB/s | 5.0 MB 00:00 2026-02-19T20:06:37.778 INFO:teuthology.orchestra.run.trial080.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5336.g80 1.3 MB/s | 57 kB 00:00 2026-02-19T20:06:37.806 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:45 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:37.825 INFO:teuthology.orchestra.run.trial080.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5336.g802bf111. 3.8 MB/s | 184 kB 00:00 2026-02-19T20:06:37.871 INFO:teuthology.orchestra.run.trial073.stdout:Package libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:37.889 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:37.890 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:37.890 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:37.911 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install libcephfs-devel 2026-02-19T20:06:38.213 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:46 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:38.276 INFO:teuthology.orchestra.run.trial080.stdout:(8/96): ceph-osd-classic-20.3.0-5336.g802bf111. 22 MB/s | 16 MB 00:00 2026-02-19T20:06:38.295 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:38.296 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:38.296 INFO:teuthology.orchestra.run.trial073.stdout: Package Arch Version Repo Size 2026-02-19T20:06:38.296 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:38.296 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:06:38.296 INFO:teuthology.orchestra.run.trial073.stdout: libcephfs-devel x86_64 2:20.3.0-5336.g802bf111.el9 ceph 37 k 2026-02-19T20:06:38.296 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:38.296 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:06:38.297 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:38.297 INFO:teuthology.orchestra.run.trial073.stdout:Install 1 Package 2026-02-19T20:06:38.297 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:38.297 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 37 k 2026-02-19T20:06:38.297 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 171 k 2026-02-19T20:06:38.297 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:06:38.579 INFO:teuthology.orchestra.run.trial073.stdout:libcephfs-devel-20.3.0-5336.g802bf111.el9.x86_6 132 kB/s | 37 kB 00:00 2026-02-19T20:06:38.579 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:38.579 INFO:teuthology.orchestra.run.trial073.stdout:Total 132 kB/s | 37 kB 00:00 2026-02-19T20:06:38.580 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:06:38.582 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:06:38.582 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:06:38.584 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:06:38.584 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:06:38.606 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:06:38.622 INFO:teuthology.orchestra.run.trial073.stdout: Installing : libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:38.635 INFO:teuthology.orchestra.run.trial080.stdout:(9/96): ceph-osd-crimson-20.3.0-5336.g802bf111. 18 MB/s | 19 MB 00:01 2026-02-19T20:06:38.644 INFO:teuthology.orchestra.run.trial080.stdout:(10/96): ceph-mgr-diskprediction-local-20.3.0-5 20 MB/s | 7.4 MB 00:00 2026-02-19T20:06:38.647 INFO:teuthology.orchestra.run.trial080.stdout:(11/96): ceph-mgr-k8sevents-20.3.0-5336.g802bf1 1.8 MB/s | 22 kB 00:00 2026-02-19T20:06:38.659 INFO:teuthology.orchestra.run.trial080.stdout:(12/96): ceph-mgr-modules-core-20.3.0-5336.g802 19 MB/s | 297 kB 00:00 2026-02-19T20:06:38.668 INFO:teuthology.orchestra.run.trial080.stdout:(13/96): ceph-mgr-rook-20.3.0-5336.g802bf111.el 2.3 MB/s | 50 kB 00:00 2026-02-19T20:06:38.678 INFO:teuthology.orchestra.run.trial080.stdout:(14/96): ceph-prometheus-alerts-20.3.0-5336.g80 940 kB/s | 17 kB 00:00 2026-02-19T20:06:38.679 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:38.692 INFO:teuthology.orchestra.run.trial080.stdout:(15/96): ceph-volume-20.3.0-5336.g802bf111.el9. 12 MB/s | 301 kB 00:00 2026-02-19T20:06:38.721 INFO:teuthology.orchestra.run.trial080.stdout:(16/96): cephadm-20.3.0-5336.g802bf111.el9.noar 24 MB/s | 1.0 MB 00:00 2026-02-19T20:06:38.739 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:38.739 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:38.739 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:38.739 INFO:teuthology.orchestra.run.trial073.stdout: libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:38.739 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:38.739 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:38.765 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install librados2 2026-02-19T20:06:39.077 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:47 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:39.144 INFO:teuthology.orchestra.run.trial073.stdout:Package librados2-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:39.162 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:39.163 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:39.163 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:39.184 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install librbd1 2026-02-19T20:06:39.200 INFO:teuthology.orchestra.run.trial080.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 84 kB/s | 40 kB 00:00 2026-02-19T20:06:39.396 INFO:teuthology.orchestra.run.trial080.stdout:(18/96): ceph-mgr-dashboard-20.3.0-5336.g802bf1 32 MB/s | 51 MB 00:01 2026-02-19T20:06:39.396 INFO:teuthology.orchestra.run.trial080.stdout:(19/96): libconfig-1.7.2-9.el9.x86_64.rpm 367 kB/s | 72 kB 00:00 2026-02-19T20:06:39.496 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:47 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:39.544 INFO:teuthology.orchestra.run.trial080.stdout:(20/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 412 kB/s | 351 kB 00:00 2026-02-19T20:06:39.562 INFO:teuthology.orchestra.run.trial073.stdout:Package librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:39.581 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:39.582 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:39.582 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:39.602 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install python3-rados 2026-02-19T20:06:39.881 INFO:teuthology.orchestra.run.trial080.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 753 kB/s | 253 kB 00:00 2026-02-19T20:06:39.911 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:47 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:39.977 INFO:teuthology.orchestra.run.trial073.stdout:Package python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:39.995 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:39.996 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:39.996 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:40.018 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install python3-rgw 2026-02-19T20:06:40.053 INFO:teuthology.orchestra.run.trial080.stdout:(22/96): libquadmath-11.5.0-14.el9.x86_64.rpm 281 kB/s | 184 kB 00:00 2026-02-19T20:06:40.231 INFO:teuthology.orchestra.run.trial080.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 597 kB/s | 106 kB 00:00 2026-02-19T20:06:40.327 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:48 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:40.359 INFO:teuthology.orchestra.run.trial080.stdout:(24/96): libgfortran-11.5.0-14.el9.x86_64.rpm 825 kB/s | 794 kB 00:00 2026-02-19T20:06:40.393 INFO:teuthology.orchestra.run.trial073.stdout:Package python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:40.412 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:40.412 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:40.413 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:40.433 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install python3-cephfs 2026-02-19T20:06:40.487 INFO:teuthology.orchestra.run.trial080.stdout:(25/96): python3-pycparser-2.20-6.el9.noarch.rp 527 kB/s | 135 kB 00:00 2026-02-19T20:06:40.616 INFO:teuthology.orchestra.run.trial080.stdout:(26/96): python3-pyparsing-2.4.7-9.el9.noarch.r 585 kB/s | 150 kB 00:00 2026-02-19T20:06:40.666 INFO:teuthology.orchestra.run.trial080.stdout:(27/96): python3-requests-2.25.1-10.el9.noarch. 710 kB/s | 126 kB 00:00 2026-02-19T20:06:40.735 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:48 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:40.801 INFO:teuthology.orchestra.run.trial073.stdout:Package python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:40.820 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:40.820 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:40.820 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:40.842 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install python3-rbd 2026-02-19T20:06:40.922 INFO:teuthology.orchestra.run.trial080.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 710 kB/s | 182 kB 00:00 2026-02-19T20:06:40.952 INFO:teuthology.orchestra.run.trial080.stdout:(29/96): python3-urllib3-1.26.5-7.el9.noarch.rp 649 kB/s | 218 kB 00:00 2026-02-19T20:06:40.971 INFO:teuthology.orchestra.run.trial080.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 1.5 MB/s | 30 kB 00:00 2026-02-19T20:06:41.000 INFO:teuthology.orchestra.run.trial080.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 104 MB/s | 3.0 MB 00:00 2026-02-19T20:06:41.004 INFO:teuthology.orchestra.run.trial080.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 3.5 MB/s | 15 kB 00:00 2026-02-19T20:06:41.010 INFO:teuthology.orchestra.run.trial080.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 40 MB/s | 246 kB 00:00 2026-02-19T20:06:41.017 INFO:teuthology.orchestra.run.trial080.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 31 MB/s | 188 kB 00:00 2026-02-19T20:06:41.021 INFO:teuthology.orchestra.run.trial080.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 9.8 MB/s | 42 kB 00:00 2026-02-19T20:06:41.040 INFO:teuthology.orchestra.run.trial080.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 275 MB/s | 5.3 MB 00:00 2026-02-19T20:06:41.063 INFO:teuthology.orchestra.run.trial080.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 260 MB/s | 6.0 MB 00:00 2026-02-19T20:06:41.070 INFO:teuthology.orchestra.run.trial080.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 34 MB/s | 244 kB 00:00 2026-02-19T20:06:41.077 INFO:teuthology.orchestra.run.trial080.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 36 MB/s | 249 kB 00:00 2026-02-19T20:06:41.082 INFO:teuthology.orchestra.run.trial080.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 9.8 MB/s | 48 kB 00:00 2026-02-19T20:06:41.088 INFO:teuthology.orchestra.run.trial080.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 30 MB/s | 177 kB 00:00 2026-02-19T20:06:41.097 INFO:teuthology.orchestra.run.trial080.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 132 MB/s | 1.2 MB 00:00 2026-02-19T20:06:41.102 INFO:teuthology.orchestra.run.trial080.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 7.2 MB/s | 35 kB 00:00 2026-02-19T20:06:41.125 INFO:teuthology.orchestra.run.trial080.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 269 MB/s | 6.1 MB 00:00 2026-02-19T20:06:41.132 INFO:teuthology.orchestra.run.trial080.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 58 MB/s | 442 kB 00:00 2026-02-19T20:06:41.138 INFO:teuthology.orchestra.run.trial080.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 14 MB/s | 77 kB 00:00 2026-02-19T20:06:41.143 INFO:teuthology.orchestra.run.trial080.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 45 MB/s | 267 kB 00:00 2026-02-19T20:06:41.146 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:49 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:41.150 INFO:teuthology.orchestra.run.trial080.stdout:(48/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 24 MB/s | 159 kB 00:00 2026-02-19T20:06:41.157 INFO:teuthology.orchestra.run.trial080.stdout:(49/96): python3-pyasn1-modules-0.4.8-6.el9.noa 39 MB/s | 279 kB 00:00 2026-02-19T20:06:41.162 INFO:teuthology.orchestra.run.trial080.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 10 MB/s | 54 kB 00:00 2026-02-19T20:06:41.213 INFO:teuthology.orchestra.run.trial073.stdout:Package python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:41.222 INFO:teuthology.orchestra.run.trial080.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 326 MB/s | 19 MB 00:00 2026-02-19T20:06:41.227 INFO:teuthology.orchestra.run.trial080.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 8.3 MB/s | 42 kB 00:00 2026-02-19T20:06:41.231 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:41.232 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:41.232 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:41.232 INFO:teuthology.orchestra.run.trial080.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 35 MB/s | 189 kB 00:00 2026-02-19T20:06:41.237 INFO:teuthology.orchestra.run.trial080.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 17 MB/s | 90 kB 00:00 2026-02-19T20:06:41.254 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install rbd-fuse 2026-02-19T20:06:41.259 INFO:teuthology.orchestra.run.trial080.stdout:(55/96): zip-3.0-35.el9.x86_64.rpm 787 kB/s | 266 kB 00:00 2026-02-19T20:06:41.291 INFO:teuthology.orchestra.run.trial080.stdout:(56/96): lua-devel-5.4.4-4.el9.x86_64.rpm 415 kB/s | 22 kB 00:00 2026-02-19T20:06:41.370 INFO:teuthology.orchestra.run.trial080.stdout:(57/96): protobuf-compiler-3.14.0-17.el9.x86_64 7.7 MB/s | 862 kB 00:00 2026-02-19T20:06:41.400 INFO:teuthology.orchestra.run.trial080.stdout:(58/96): python3-cryptography-36.0.1-5.el9.x86_ 840 kB/s | 1.2 MB 00:01 2026-02-19T20:06:41.421 INFO:teuthology.orchestra.run.trial080.stdout:(59/96): grpc-data-1.46.7-10.el9.noarch.rpm 382 kB/s | 19 kB 00:00 2026-02-19T20:06:41.426 INFO:teuthology.orchestra.run.trial080.stdout:(60/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 4.0 MB/s | 551 kB 00:00 2026-02-19T20:06:41.441 INFO:teuthology.orchestra.run.trial080.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 1.9 MB/s | 29 kB 00:00 2026-02-19T20:06:41.456 INFO:teuthology.orchestra.run.trial080.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 3.9 MB/s | 60 kB 00:00 2026-02-19T20:06:41.471 INFO:teuthology.orchestra.run.trial080.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.9 MB/s | 43 kB 00:00 2026-02-19T20:06:41.477 INFO:teuthology.orchestra.run.trial080.stdout:(64/96): luarocks-3.9.2-5.el9.noarch.rpm 1.9 MB/s | 151 kB 00:00 2026-02-19T20:06:41.495 INFO:teuthology.orchestra.run.trial080.stdout:(65/96): python3-cachetools-4.2.4-1.el9.noarch. 1.3 MB/s | 32 kB 00:00 2026-02-19T20:06:41.507 INFO:teuthology.orchestra.run.trial080.stdout:(66/96): python3-certifi-2023.05.07-4.el9.noarc 462 kB/s | 14 kB 00:00 2026-02-19T20:06:41.512 INFO:teuthology.orchestra.run.trial080.stdout:(67/96): python3-cheroot-10.0.1-4.el9.noarch.rp 10 MB/s | 173 kB 00:00 2026-02-19T20:06:41.520 INFO:teuthology.orchestra.run.trial080.stdout:(68/96): python3-asyncssh-2.13.2-5.el9.noarch.r 5.4 MB/s | 548 kB 00:00 2026-02-19T20:06:41.546 INFO:teuthology.orchestra.run.trial080.stdout:(69/96): python3-cherrypy-18.6.1-2.el9.noarch.r 9.1 MB/s | 358 kB 00:00 2026-02-19T20:06:41.551 INFO:teuthology.orchestra.run.trial080.stdout:(70/96): python3-google-auth-2.45.0-1.el9.noarc 6.4 MB/s | 254 kB 00:00 2026-02-19T20:06:41.558 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:49 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:41.574 INFO:teuthology.orchestra.run.trial080.stdout:(71/96): python3-grpcio-tools-1.46.7-10.el9.x86 5.1 MB/s | 144 kB 00:00 2026-02-19T20:06:41.576 INFO:teuthology.orchestra.run.trial080.stdout:(72/96): python3-influxdb-5.3.1-1.el9.noarch.rp 5.4 MB/s | 139 kB 00:00 2026-02-19T20:06:41.591 INFO:teuthology.orchestra.run.trial080.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 746 kB/s | 11 kB 00:00 2026-02-19T20:06:41.601 INFO:teuthology.orchestra.run.trial080.stdout:(74/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 25 MB/s | 2.0 MB 00:00 2026-02-19T20:06:41.601 INFO:teuthology.orchestra.run.trial080.stdout:(75/96): python3-isodate-0.6.1-3.el9.noarch.rpm 2.0 MB/s | 56 kB 00:00 2026-02-19T20:06:41.605 INFO:teuthology.orchestra.run.trial080.stdout:(76/96): python3-jaraco-classes-3.2.1-5.el9.noa 1.2 MB/s | 18 kB 00:00 2026-02-19T20:06:41.617 INFO:teuthology.orchestra.run.trial080.stdout:(77/96): python3-jaraco-context-6.0.1-3.el9.noa 1.3 MB/s | 20 kB 00:00 2026-02-19T20:06:41.617 INFO:teuthology.orchestra.run.trial080.stdout:(78/96): python3-jaraco-collections-3.0.0-8.el9 1.4 MB/s | 23 kB 00:00 2026-02-19T20:06:41.620 INFO:teuthology.orchestra.run.trial080.stdout:(79/96): python3-jaraco-functools-3.5.0-2.el9.n 1.3 MB/s | 19 kB 00:00 2026-02-19T20:06:41.632 INFO:teuthology.orchestra.run.trial080.stdout:(80/96): python3-jaraco-text-4.0.0-2.el9.noarch 1.7 MB/s | 26 kB 00:00 2026-02-19T20:06:41.635 INFO:teuthology.orchestra.run.trial080.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 5.1 MB/s | 79 kB 00:00 2026-02-19T20:06:41.642 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout: Package Architecture Version Repository Size 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout: rbd-fuse x86_64 2:20.3.0-5336.g802bf111.el9 ceph 85 k 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout:Install 1 Package 2026-02-19T20:06:41.643 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:41.644 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 85 k 2026-02-19T20:06:41.644 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 217 k 2026-02-19T20:06:41.644 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:06:41.661 INFO:teuthology.orchestra.run.trial080.stdout:(82/96): python3-kubernetes-26.1.0-3.el9.noarch 24 MB/s | 1.0 MB 00:00 2026-02-19T20:06:41.662 INFO:teuthology.orchestra.run.trial080.stdout:(83/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.9 MB/s | 86 kB 00:00 2026-02-19T20:06:41.671 INFO:teuthology.orchestra.run.trial080.stdout:(84/96): python3-natsort-7.1.1-5.el9.noarch.rpm 1.6 MB/s | 58 kB 00:00 2026-02-19T20:06:41.676 INFO:teuthology.orchestra.run.trial080.stdout:(85/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 5.8 MB/s | 90 kB 00:00 2026-02-19T20:06:41.677 INFO:teuthology.orchestra.run.trial080.stdout:(86/96): python3-portend-3.1.0-2.el9.noarch.rpm 1.0 MB/s | 16 kB 00:00 2026-02-19T20:06:41.693 INFO:teuthology.orchestra.run.trial080.stdout:(87/96): python3-repoze-lru-0.7-16.el9.noarch.r 1.3 MB/s | 31 kB 00:00 2026-02-19T20:06:41.698 INFO:teuthology.orchestra.run.trial080.stdout:(88/96): python3-routes-2.5.1-5.el9.noarch.rpm 8.4 MB/s | 188 kB 00:00 2026-02-19T20:06:41.705 INFO:teuthology.orchestra.run.trial080.stdout:(89/96): python3-rsa-4.9-2.el9.noarch.rpm 2.0 MB/s | 59 kB 00:00 2026-02-19T20:06:41.713 INFO:teuthology.orchestra.run.trial080.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 2.3 MB/s | 36 kB 00:00 2026-02-19T20:06:41.720 INFO:teuthology.orchestra.run.trial073.stdout:rbd-fuse-20.3.0-5336.g802bf111.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-19T20:06:41.721 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:41.721 INFO:teuthology.orchestra.run.trial073.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-19T20:06:41.721 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:06:41.724 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:06:41.725 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:06:41.725 INFO:teuthology.orchestra.run.trial080.stdout:(91/96): python3-saml-1.16.0-1.el9.noarch.rpm 3.9 MB/s | 125 kB 00:00 2026-02-19T20:06:41.733 INFO:teuthology.orchestra.run.trial080.stdout:(92/96): python3-typing-extensions-4.15.0-1.el9 3.0 MB/s | 86 kB 00:00 2026-02-19T20:06:41.740 INFO:teuthology.orchestra.run.trial080.stdout:(93/96): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2026-02-19T20:06:41.755 INFO:teuthology.orchestra.run.trial080.stdout:(94/96): python3-zc-lockfile-2.0-10.el9.noarch. 1.3 MB/s | 20 kB 00:00 2026-02-19T20:06:41.756 INFO:teuthology.orchestra.run.trial080.stdout:(95/96): python3-xmltodict-0.12.0-15.el9.noarch 983 kB/s | 22 kB 00:00 2026-02-19T20:06:41.756 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:06:41.756 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:06:41.776 INFO:teuthology.orchestra.run.trial080.stdout:(96/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 953 kB/s | 48 kB 00:00 2026-02-19T20:06:41.779 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:41.779 INFO:teuthology.orchestra.run.trial080.stdout:Total 32 MB/s | 159 MB 00:04 2026-02-19T20:06:41.808 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:06:41.825 INFO:teuthology.orchestra.run.trial073.stdout: Installing : rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:41.905 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:41.962 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:41.963 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:41.963 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:41.963 INFO:teuthology.orchestra.run.trial073.stdout: rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:41.963 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:41.963 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:41.992 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install rbd-mirror 2026-02-19T20:06:42.213 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:06:42.252 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:06:42.253 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:06:42.303 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:50 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:42.386 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:42.386 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:42.387 INFO:teuthology.orchestra.run.trial073.stdout: Package Arch Version Repo Size 2026-02-19T20:06:42.387 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:42.387 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:06:42.387 INFO:teuthology.orchestra.run.trial073.stdout: rbd-mirror x86_64 2:20.3.0-5336.g802bf111.el9 ceph 2.9 M 2026-02-19T20:06:42.387 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:42.387 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:06:42.388 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:42.388 INFO:teuthology.orchestra.run.trial073.stdout:Install 1 Package 2026-02-19T20:06:42.388 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:42.388 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 2.9 M 2026-02-19T20:06:42.388 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 11 M 2026-02-19T20:06:42.388 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:06:42.537 INFO:teuthology.orchestra.run.trial073.stdout:rbd-mirror-20.3.0-5336.g802bf111.el9.x86_64.rpm 19 MB/s | 2.9 MB 00:00 2026-02-19T20:06:42.537 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:42.538 INFO:teuthology.orchestra.run.trial073.stdout:Total 19 MB/s | 2.9 MB 00:00 2026-02-19T20:06:42.538 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:06:42.542 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:06:42.542 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:06:42.581 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:06:42.581 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:06:42.756 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:06:42.758 INFO:teuthology.orchestra.run.trial073.stdout: Installing : rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:42.781 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:42.782 INFO:teuthology.orchestra.run.trial073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:42.782 INFO:teuthology.orchestra.run.trial073.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-19T20:06:42.782 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-19T20:06:42.782 INFO:teuthology.orchestra.run.trial073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-19T20:06:42.782 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:42.856 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:06:42.856 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:06:43.068 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:43.069 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:43.069 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:43.069 INFO:teuthology.orchestra.run.trial073.stdout: rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:43.069 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:43.069 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:43.098 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install rbd-nbd 2026-02-19T20:06:43.412 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:51 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:43.496 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:43.496 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:43.496 INFO:teuthology.orchestra.run.trial073.stdout: Package Architecture Version Repository Size 2026-02-19T20:06:43.496 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:43.496 INFO:teuthology.orchestra.run.trial073.stdout:Installing: 2026-02-19T20:06:43.497 INFO:teuthology.orchestra.run.trial073.stdout: rbd-nbd x86_64 2:20.3.0-5336.g802bf111.el9 ceph 173 k 2026-02-19T20:06:43.497 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:43.497 INFO:teuthology.orchestra.run.trial073.stdout:Transaction Summary 2026-02-19T20:06:43.497 INFO:teuthology.orchestra.run.trial073.stdout:================================================================================ 2026-02-19T20:06:43.497 INFO:teuthology.orchestra.run.trial073.stdout:Install 1 Package 2026-02-19T20:06:43.497 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:43.497 INFO:teuthology.orchestra.run.trial073.stdout:Total download size: 173 k 2026-02-19T20:06:43.498 INFO:teuthology.orchestra.run.trial073.stdout:Installed size: 466 k 2026-02-19T20:06:43.498 INFO:teuthology.orchestra.run.trial073.stdout:Downloading Packages: 2026-02-19T20:06:43.498 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:06:43.500 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-19T20:06:43.504 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-19T20:06:43.525 INFO:teuthology.orchestra.run.trial080.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-19T20:06:43.558 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-19T20:06:43.564 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-19T20:06:43.567 INFO:teuthology.orchestra.run.trial080.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-19T20:06:43.571 INFO:teuthology.orchestra.run.trial080.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-19T20:06:43.582 INFO:teuthology.orchestra.run.trial073.stdout:rbd-nbd-20.3.0-5336.g802bf111.el9.x86_64.rpm 2.0 MB/s | 173 kB 00:00 2026-02-19T20:06:43.583 INFO:teuthology.orchestra.run.trial073.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:43.583 INFO:teuthology.orchestra.run.trial073.stdout:Total 2.0 MB/s | 173 kB 00:00 2026-02-19T20:06:43.583 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction check 2026-02-19T20:06:43.587 INFO:teuthology.orchestra.run.trial073.stdout:Transaction check succeeded. 2026-02-19T20:06:43.587 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction test 2026-02-19T20:06:43.594 INFO:teuthology.orchestra.run.trial080.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-19T20:06:43.607 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-19T20:06:43.610 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-19T20:06:43.616 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-19T20:06:43.618 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-19T20:06:43.619 INFO:teuthology.orchestra.run.trial073.stdout:Transaction test succeeded. 2026-02-19T20:06:43.619 INFO:teuthology.orchestra.run.trial073.stdout:Running transaction 2026-02-19T20:06:43.629 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-19T20:06:43.636 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-19T20:06:43.660 INFO:teuthology.orchestra.run.trial080.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-19T20:06:43.663 INFO:teuthology.orchestra.run.trial080.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-19T20:06:43.675 INFO:teuthology.orchestra.run.trial073.stdout: Preparing : 1/1 2026-02-19T20:06:43.692 INFO:teuthology.orchestra.run.trial073.stdout: Installing : rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:43.714 INFO:teuthology.orchestra.run.trial080.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-19T20:06:43.727 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-19T20:06:43.729 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-19T20:06:43.733 INFO:teuthology.orchestra.run.trial080.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-19T20:06:43.739 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-19T20:06:43.743 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-19T20:06:43.746 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-19T20:06:43.761 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-19T20:06:43.765 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-19T20:06:43.770 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-19T20:06:43.772 INFO:teuthology.orchestra.run.trial073.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:43.782 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-19T20:06:43.790 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-19T20:06:43.797 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-19T20:06:43.816 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-19T20:06:43.827 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-19T20:06:43.831 INFO:teuthology.orchestra.run.trial073.stdout: Verifying : rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:43.831 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:43.831 INFO:teuthology.orchestra.run.trial073.stdout:Installed: 2026-02-19T20:06:43.831 INFO:teuthology.orchestra.run.trial073.stdout: rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:43.831 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:06:43.832 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:43.843 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-19T20:06:43.847 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-19T20:06:43.855 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-19T20:06:43.860 DEBUG:teuthology.orchestra.run.trial073:> sudo yum -y install rbd-mirror 2026-02-19T20:06:43.895 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-19T20:06:44.169 INFO:teuthology.orchestra.run.trial073.stdout:Last metadata expiration check: 0:00:52 ago on Thu 19 Feb 2026 08:05:52 PM UTC. 2026-02-19T20:06:44.219 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-19T20:06:44.229 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-19T20:06:44.233 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-19T20:06:44.236 INFO:teuthology.orchestra.run.trial073.stdout:Package rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:44.239 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-19T20:06:44.241 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-19T20:06:44.254 INFO:teuthology.orchestra.run.trial073.stdout:Dependencies resolved. 2026-02-19T20:06:44.255 INFO:teuthology.orchestra.run.trial073.stdout:Nothing to do. 2026-02-19T20:06:44.255 INFO:teuthology.orchestra.run.trial073.stdout:Complete! 2026-02-19T20:06:44.268 INFO:teuthology.orchestra.run.trial080.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-19T20:06:44.280 DEBUG:teuthology.parallel:result is None 2026-02-19T20:06:44.312 INFO:teuthology.orchestra.run.trial080.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-19T20:06:44.321 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-19T20:06:44.326 INFO:teuthology.orchestra.run.trial080.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-19T20:06:44.330 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-19T20:06:44.335 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-19T20:06:44.339 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-19T20:06:44.346 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-19T20:06:44.349 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-19T20:06:44.377 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-19T20:06:44.387 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-19T20:06:44.393 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-19T20:06:44.400 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-19T20:06:44.435 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-19T20:06:44.661 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-19T20:06:44.685 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-19T20:06:44.689 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-19T20:06:44.748 INFO:teuthology.orchestra.run.trial080.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-19T20:06:44.750 INFO:teuthology.orchestra.run.trial080.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-19T20:06:44.771 INFO:teuthology.orchestra.run.trial080.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-19T20:06:45.100 INFO:teuthology.orchestra.run.trial080.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-19T20:06:45.174 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-19T20:06:45.854 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-19T20:06:45.870 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-19T20:06:45.880 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-19T20:06:45.887 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-19T20:06:45.900 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-19T20:06:45.916 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-19T20:06:45.991 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-19T20:06:46.000 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-19T20:06:46.023 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-19T20:06:46.053 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-19T20:06:46.103 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-19T20:06:46.109 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-19T20:06:46.113 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-19T20:06:46.114 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-19T20:06:46.133 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-19T20:06:46.133 INFO:teuthology.orchestra.run.trial080.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-19T20:06:46.133 INFO:teuthology.orchestra.run.trial080.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-19T20:06:46.133 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:46.142 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-19T20:06:46.170 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-19T20:06:46.170 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-19T20:06:46.170 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:46.204 INFO:teuthology.orchestra.run.trial080.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-19T20:06:46.829 INFO:teuthology.orchestra.run.trial080.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-19T20:06:47.600 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_6 80/96 2026-02-19T20:06:47.664 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_6 81/96 2026-02-19T20:06:47.666 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 82/96 2026-02-19T20:06:47.680 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 82/96 2026-02-19T20:06:47.680 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:47.681 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-19T20:06:47.681 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:47.691 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 83/96 2026-02-19T20:06:47.715 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 83/96 2026-02-19T20:06:47.715 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:47.715 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-19T20:06:47.715 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-19T20:06:47.715 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-19T20:06:47.715 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:47.776 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: cephadm-2:20.3.0-5336.g802bf111.el9.noarch 84/96 2026-02-19T20:06:47.778 INFO:teuthology.orchestra.run.trial080.stdout: Installing : cephadm-2:20.3.0-5336.g802bf111.el9.noarch 84/96 2026-02-19T20:06:47.787 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5336.g802bf111.el9 85/96 2026-02-19T20:06:47.812 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5336.g802bf111.el 86/96 2026-02-19T20:06:47.813 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarc 87/96 2026-02-19T20:06:50.709 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarc 87/96 2026-02-19T20:06:50.716 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noa 88/96 2026-02-19T20:06:51.175 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noa 88/96 2026-02-19T20:06:51.176 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf 89/96 2026-02-19T20:06:51.188 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf 89/96 2026-02-19T20:06:51.189 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noa 90/96 2026-02-19T20:06:51.251 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noa 90/96 2026-02-19T20:06:51.293 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5336.g802bf111.el9. 91/96 2026-02-19T20:06:51.294 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 92/96 2026-02-19T20:06:51.316 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 92/96 2026-02-19T20:06:51.316 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:51.316 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-19T20:06:51.316 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-19T20:06:51.316 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-19T20:06:51.316 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:51.328 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 93/96 2026-02-19T20:06:51.551 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 93/96 2026-02-19T20:06:51.552 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 94/96 2026-02-19T20:06:51.574 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 94/96 2026-02-19T20:06:51.574 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:51.574 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-19T20:06:51.574 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-19T20:06:51.574 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-19T20:06:51.574 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:51.671 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 95/96 2026-02-19T20:06:51.692 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 95/96 2026-02-19T20:06:51.692 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:51.692 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-19T20:06:51.693 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-19T20:06:51.693 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-19T20:06:51.693 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:51.694 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-2:20.3.0-5336.g802bf111.el9.x86_64 96/96 2026-02-19T20:06:51.700 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_6 96/96 2026-02-19T20:06:51.725 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_6 96/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-2:20.3.0-5336.g802bf111.el9.x86_64 96/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-2:20.3.0-5336.g802bf111.el9.x86_64 1/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 2/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 3/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 4/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 5/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_6 6/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_6 7/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5336.g802bf111.el 8/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarc 9/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noa 10/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf 11/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noa 12/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5336.g802bf111.el9. 13/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 14/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5336.g802bf111.el9 15/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 16/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : cephadm-2:20.3.0-5336.g802bf111.el9.noarch 17/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-19T20:06:52.513 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-19T20:06:52.514 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-19T20:06:52.515 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-grafana-dashboards-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mds-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-k8sevents-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-modules-core-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-mon-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-osd-crimson-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-prometheus-alerts-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: cephadm-2:20.3.0-5336.g802bf111.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-19T20:06:52.580 INFO:teuthology.orchestra.run.trial080.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: lua-5.4.4-4.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-19T20:06:52.581 INFO:teuthology.orchestra.run.trial080.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-ply-3.11-14.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: unzip-6.0-59.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: zip-3.0-35.el9.x86_64 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:52.582 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:52.644 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-base 2026-02-19T20:06:52.956 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:38 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:53.021 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-base-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:53.039 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:53.040 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:53.040 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:53.062 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install cephadm 2026-02-19T20:06:53.370 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:39 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:53.435 INFO:teuthology.orchestra.run.trial080.stdout:Package cephadm-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:53.453 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:53.454 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:53.454 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:53.475 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-immutable-object-cache 2026-02-19T20:06:53.788 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:39 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:53.871 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:53.872 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:53.872 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-19T20:06:53.872 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:53.872 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:06:53.872 INFO:teuthology.orchestra.run.trial080.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5336.g802bf111.el9 ceph 147 k 2026-02-19T20:06:53.872 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:53.872 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:06:53.873 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:53.873 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-19T20:06:53.873 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:53.873 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 147 k 2026-02-19T20:06:53.873 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 423 k 2026-02-19T20:06:53.873 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:06:54.082 INFO:teuthology.orchestra.run.trial080.stdout:ceph-immutable-object-cache-20.3.0-5336.g802bf1 703 kB/s | 147 kB 00:00 2026-02-19T20:06:54.083 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:54.083 INFO:teuthology.orchestra.run.trial080.stdout:Total 699 kB/s | 147 kB 00:00 2026-02-19T20:06:54.083 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:06:54.087 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:06:54.087 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:06:54.118 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:06:54.118 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:06:54.173 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:06:54.175 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5336.g802bf111. 1/1 2026-02-19T20:06:54.198 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5336.g802bf111. 1/1 2026-02-19T20:06:54.198 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:06:54.198 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-19T20:06:54.199 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:54.511 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5336.g802bf111. 1/1 2026-02-19T20:06:54.512 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:54.512 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:06:54.512 INFO:teuthology.orchestra.run.trial080.stdout: ceph-immutable-object-cache-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:54.512 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:54.512 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:54.539 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr 2026-02-19T20:06:54.848 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:40 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:54.914 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:54.932 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:54.933 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:54.933 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:54.953 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr-dashboard 2026-02-19T20:06:55.254 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:41 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:55.319 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-dashboard-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:55.337 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:55.337 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:55.338 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:55.358 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-19T20:06:55.669 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:41 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:55.734 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:55.752 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:55.753 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:55.753 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:55.773 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr-rook 2026-02-19T20:06:56.084 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:42 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:56.148 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-rook-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:56.166 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:56.167 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:56.167 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:56.188 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-mgr-cephadm 2026-02-19T20:06:56.492 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:42 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:56.558 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-mgr-cephadm-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:56.576 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:56.577 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:56.577 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:56.598 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-osd 2026-02-19T20:06:56.907 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:42 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:56.972 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-osd-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:56.991 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:56.991 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:56.992 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:57.013 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-osd-classic 2026-02-19T20:06:57.323 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:43 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:57.388 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-osd-classic-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:06:57.406 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:57.407 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:57.407 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:57.428 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-fuse 2026-02-19T20:06:57.736 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:43 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:57.820 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:57.820 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:57.820 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-19T20:06:57.820 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:57.820 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:06:57.820 INFO:teuthology.orchestra.run.trial080.stdout: ceph-fuse x86_64 2:20.3.0-5336.g802bf111.el9 ceph 987 k 2026-02-19T20:06:57.821 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:57.821 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:06:57.821 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:57.821 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-19T20:06:57.821 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:57.821 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 987 k 2026-02-19T20:06:57.821 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 2.9 M 2026-02-19T20:06:57.821 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:06:57.975 INFO:teuthology.orchestra.run.trial080.stdout:ceph-fuse-20.3.0-5336.g802bf111.el9.x86_64.rpm 6.2 MB/s | 987 kB 00:00 2026-02-19T20:06:57.976 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:57.976 INFO:teuthology.orchestra.run.trial080.stdout:Total 6.2 MB/s | 987 kB 00:00 2026-02-19T20:06:57.976 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:06:57.980 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:06:57.981 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:06:58.015 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:06:58.016 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:06:58.107 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:06:58.124 INFO:teuthology.orchestra.run.trial080.stdout: Installing : ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:58.372 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:58.429 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:58.429 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:58.429 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:06:58.429 INFO:teuthology.orchestra.run.trial080.stdout: ceph-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:58.429 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:58.429 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:58.456 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install ceph-volume 2026-02-19T20:06:58.775 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:44 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:58.840 INFO:teuthology.orchestra.run.trial080.stdout:Package ceph-volume-2:20.3.0-5336.g802bf111.el9.noarch is already installed. 2026-02-19T20:06:58.858 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:58.859 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:06:58.859 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:58.881 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install librados-devel 2026-02-19T20:06:59.190 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:45 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:06:59.274 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:06:59.274 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:59.274 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-19T20:06:59.274 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout: librados-devel x86_64 2:20.3.0-5336.g802bf111.el9 ceph 123 k 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 123 k 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 437 k 2026-02-19T20:06:59.275 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:06:59.360 INFO:teuthology.orchestra.run.trial080.stdout:librados-devel-20.3.0-5336.g802bf111.el9.x86_64 1.4 MB/s | 123 kB 00:00 2026-02-19T20:06:59.360 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:06:59.360 INFO:teuthology.orchestra.run.trial080.stdout:Total 1.4 MB/s | 123 kB 00:00 2026-02-19T20:06:59.360 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:06:59.363 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:06:59.363 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:06:59.394 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:06:59.394 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:06:59.448 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:06:59.465 INFO:teuthology.orchestra.run.trial080.stdout: Installing : librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:59.546 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:59.604 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:06:59.604 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:59.604 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:06:59.604 INFO:teuthology.orchestra.run.trial080.stdout: librados-devel-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:06:59.604 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:06:59.604 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:06:59.633 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install libcephfs2 2026-02-19T20:06:59.952 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:45 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:00.017 INFO:teuthology.orchestra.run.trial080.stdout:Package libcephfs2-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:07:00.035 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:00.036 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:07:00.036 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:00.058 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install libcephfs-devel 2026-02-19T20:07:00.368 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:46 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:00.450 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:00.451 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs-devel x86_64 2:20.3.0-5336.g802bf111.el9 ceph 37 k 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:00.452 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 37 k 2026-02-19T20:07:00.453 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 171 k 2026-02-19T20:07:00.453 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:07:00.520 INFO:teuthology.orchestra.run.trial080.stdout:libcephfs-devel-20.3.0-5336.g802bf111.el9.x86_6 541 kB/s | 37 kB 00:00 2026-02-19T20:07:00.521 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:07:00.521 INFO:teuthology.orchestra.run.trial080.stdout:Total 532 kB/s | 37 kB 00:00 2026-02-19T20:07:00.522 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:07:00.524 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:07:00.524 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:07:00.526 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:07:00.526 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:07:00.547 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:07:00.563 INFO:teuthology.orchestra.run.trial080.stdout: Installing : libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:00.619 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:00.675 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:00.676 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:00.676 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:07:00.676 INFO:teuthology.orchestra.run.trial080.stdout: libcephfs-devel-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:07:00.676 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:00.676 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:00.701 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install librados2 2026-02-19T20:07:01.011 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:47 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:01.076 INFO:teuthology.orchestra.run.trial080.stdout:Package librados2-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:07:01.094 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:01.095 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:07:01.095 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:01.117 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install librbd1 2026-02-19T20:07:01.421 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:47 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:01.485 INFO:teuthology.orchestra.run.trial080.stdout:Package librbd1-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:07:01.504 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:01.504 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:07:01.505 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:01.525 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-rados 2026-02-19T20:07:01.834 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:47 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:01.899 INFO:teuthology.orchestra.run.trial080.stdout:Package python3-rados-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:07:01.917 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:01.918 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:07:01.918 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:01.940 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-rgw 2026-02-19T20:07:02.249 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:48 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:02.313 INFO:teuthology.orchestra.run.trial080.stdout:Package python3-rgw-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:07:02.332 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:02.333 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:07:02.333 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:02.355 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-cephfs 2026-02-19T20:07:02.665 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:48 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:02.729 INFO:teuthology.orchestra.run.trial080.stdout:Package python3-cephfs-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:07:02.747 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:02.748 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:07:02.748 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:02.770 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install python3-rbd 2026-02-19T20:07:03.080 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:49 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:03.144 INFO:teuthology.orchestra.run.trial080.stdout:Package python3-rbd-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:07:03.162 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:03.163 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:07:03.163 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:03.185 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-fuse 2026-02-19T20:07:03.493 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:49 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:03.576 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:03.577 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:03.577 INFO:teuthology.orchestra.run.trial080.stdout: Package Architecture Version Repository Size 2026-02-19T20:07:03.577 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:03.577 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:07:03.577 INFO:teuthology.orchestra.run.trial080.stdout: rbd-fuse x86_64 2:20.3.0-5336.g802bf111.el9 ceph 85 k 2026-02-19T20:07:03.577 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:03.578 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:07:03.578 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:03.578 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-19T20:07:03.578 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:03.578 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 85 k 2026-02-19T20:07:03.578 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 217 k 2026-02-19T20:07:03.578 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:07:03.652 INFO:teuthology.orchestra.run.trial080.stdout:rbd-fuse-20.3.0-5336.g802bf111.el9.x86_64.rpm 1.1 MB/s | 85 kB 00:00 2026-02-19T20:07:03.652 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:07:03.652 INFO:teuthology.orchestra.run.trial080.stdout:Total 1.1 MB/s | 85 kB 00:00 2026-02-19T20:07:03.652 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:07:03.656 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:07:03.656 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:07:03.689 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:07:03.689 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:07:03.740 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:07:03.757 INFO:teuthology.orchestra.run.trial080.stdout: Installing : rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:03.839 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:03.897 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:03.897 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:03.898 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:07:03.898 INFO:teuthology.orchestra.run.trial080.stdout: rbd-fuse-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:07:03.898 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:03.898 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:03.928 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-mirror 2026-02-19T20:07:04.240 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:50 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:04.324 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:04.325 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:04.325 INFO:teuthology.orchestra.run.trial080.stdout: Package Arch Version Repo Size 2026-02-19T20:07:04.325 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:04.325 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:07:04.325 INFO:teuthology.orchestra.run.trial080.stdout: rbd-mirror x86_64 2:20.3.0-5336.g802bf111.el9 ceph 2.9 M 2026-02-19T20:07:04.326 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:04.326 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:07:04.326 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:04.326 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-19T20:07:04.326 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:04.326 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 2.9 M 2026-02-19T20:07:04.326 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 11 M 2026-02-19T20:07:04.326 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:07:04.472 INFO:teuthology.orchestra.run.trial080.stdout:rbd-mirror-20.3.0-5336.g802bf111.el9.x86_64.rpm 20 MB/s | 2.9 MB 00:00 2026-02-19T20:07:04.472 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:07:04.473 INFO:teuthology.orchestra.run.trial080.stdout:Total 19 MB/s | 2.9 MB 00:00 2026-02-19T20:07:04.473 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:07:04.477 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:07:04.477 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:07:04.516 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:07:04.517 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:07:04.695 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:07:04.697 INFO:teuthology.orchestra.run.trial080.stdout: Installing : rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:04.718 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:04.718 INFO:teuthology.orchestra.run.trial080.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-19T20:07:04.718 INFO:teuthology.orchestra.run.trial080.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-19T20:07:04.718 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-19T20:07:04.718 INFO:teuthology.orchestra.run.trial080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-19T20:07:04.718 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:05.007 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:05.008 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:05.008 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:07:05.008 INFO:teuthology.orchestra.run.trial080.stdout: rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:07:05.008 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:05.008 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:05.036 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-nbd 2026-02-19T20:07:05.350 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:51 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:05.434 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:05.434 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:05.434 INFO:teuthology.orchestra.run.trial080.stdout: Package Architecture Version Repository Size 2026-02-19T20:07:05.434 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:05.434 INFO:teuthology.orchestra.run.trial080.stdout:Installing: 2026-02-19T20:07:05.434 INFO:teuthology.orchestra.run.trial080.stdout: rbd-nbd x86_64 2:20.3.0-5336.g802bf111.el9 ceph 173 k 2026-02-19T20:07:05.435 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:05.435 INFO:teuthology.orchestra.run.trial080.stdout:Transaction Summary 2026-02-19T20:07:05.435 INFO:teuthology.orchestra.run.trial080.stdout:================================================================================ 2026-02-19T20:07:05.435 INFO:teuthology.orchestra.run.trial080.stdout:Install 1 Package 2026-02-19T20:07:05.435 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:05.435 INFO:teuthology.orchestra.run.trial080.stdout:Total download size: 173 k 2026-02-19T20:07:05.435 INFO:teuthology.orchestra.run.trial080.stdout:Installed size: 466 k 2026-02-19T20:07:05.435 INFO:teuthology.orchestra.run.trial080.stdout:Downloading Packages: 2026-02-19T20:07:05.520 INFO:teuthology.orchestra.run.trial080.stdout:rbd-nbd-20.3.0-5336.g802bf111.el9.x86_64.rpm 2.0 MB/s | 173 kB 00:00 2026-02-19T20:07:05.520 INFO:teuthology.orchestra.run.trial080.stdout:-------------------------------------------------------------------------------- 2026-02-19T20:07:05.520 INFO:teuthology.orchestra.run.trial080.stdout:Total 2.0 MB/s | 173 kB 00:00 2026-02-19T20:07:05.520 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction check 2026-02-19T20:07:05.525 INFO:teuthology.orchestra.run.trial080.stdout:Transaction check succeeded. 2026-02-19T20:07:05.525 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction test 2026-02-19T20:07:05.557 INFO:teuthology.orchestra.run.trial080.stdout:Transaction test succeeded. 2026-02-19T20:07:05.557 INFO:teuthology.orchestra.run.trial080.stdout:Running transaction 2026-02-19T20:07:05.614 INFO:teuthology.orchestra.run.trial080.stdout: Preparing : 1/1 2026-02-19T20:07:05.630 INFO:teuthology.orchestra.run.trial080.stdout: Installing : rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:05.715 INFO:teuthology.orchestra.run.trial080.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:05.773 INFO:teuthology.orchestra.run.trial080.stdout: Verifying : rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 1/1 2026-02-19T20:07:05.773 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:05.774 INFO:teuthology.orchestra.run.trial080.stdout:Installed: 2026-02-19T20:07:05.774 INFO:teuthology.orchestra.run.trial080.stdout: rbd-nbd-2:20.3.0-5336.g802bf111.el9.x86_64 2026-02-19T20:07:05.774 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:05.774 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:05.803 DEBUG:teuthology.orchestra.run.trial080:> sudo yum -y install rbd-mirror 2026-02-19T20:07:06.113 INFO:teuthology.orchestra.run.trial080.stdout:Last metadata expiration check: 0:00:52 ago on Thu 19 Feb 2026 08:06:14 PM UTC. 2026-02-19T20:07:06.177 INFO:teuthology.orchestra.run.trial080.stdout:Package rbd-mirror-2:20.3.0-5336.g802bf111.el9.x86_64 is already installed. 2026-02-19T20:07:06.196 INFO:teuthology.orchestra.run.trial080.stdout:Dependencies resolved. 2026-02-19T20:07:06.196 INFO:teuthology.orchestra.run.trial080.stdout:Nothing to do. 2026-02-19T20:07:06.197 INFO:teuthology.orchestra.run.trial080.stdout:Complete! 2026-02-19T20:07:06.221 DEBUG:teuthology.parallel:result is None 2026-02-19T20:07:06.221 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T20:07:06.396 DEBUG:teuthology.orchestra.run.trial073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-19T20:07:06.417 INFO:teuthology.orchestra.run.trial073.stdout:20.3.0-5336.g802bf111.el9 2026-02-19T20:07:06.417 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5336.g802bf111.el9 2026-02-19T20:07:06.418 INFO:teuthology.task.install:The correct ceph version 20.3.0-5336.g802bf111 is installed. 2026-02-19T20:07:06.419 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T20:07:06.580 DEBUG:teuthology.orchestra.run.trial080:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-19T20:07:06.601 INFO:teuthology.orchestra.run.trial080.stdout:20.3.0-5336.g802bf111.el9 2026-02-19T20:07:06.601 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5336.g802bf111.el9 2026-02-19T20:07:06.601 INFO:teuthology.task.install:The correct ceph version 20.3.0-5336.g802bf111 is installed. 2026-02-19T20:07:06.604 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-19T20:07:06.604 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:06.604 DEBUG:teuthology.orchestra.run.trial073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-19T20:07:06.631 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:06.631 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-19T20:07:06.668 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-19T20:07:06.669 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:06.669 DEBUG:teuthology.orchestra.run.trial073:> sudo dd of=/usr/bin/daemon-helper 2026-02-19T20:07:06.697 DEBUG:teuthology.orchestra.run.trial073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-19T20:07:06.762 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:06.762 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/usr/bin/daemon-helper 2026-02-19T20:07:06.788 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-19T20:07:06.854 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-19T20:07:06.854 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:06.854 DEBUG:teuthology.orchestra.run.trial073:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-19T20:07:06.880 DEBUG:teuthology.orchestra.run.trial073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-19T20:07:06.944 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:06.944 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-19T20:07:06.969 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-19T20:07:07.033 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-19T20:07:07.034 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:07.034 DEBUG:teuthology.orchestra.run.trial073:> sudo dd of=/usr/bin/stdin-killer 2026-02-19T20:07:07.059 DEBUG:teuthology.orchestra.run.trial073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-19T20:07:07.123 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:07.124 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/usr/bin/stdin-killer 2026-02-19T20:07:07.149 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-19T20:07:07.213 INFO:teuthology.run_tasks:Running task ceph... 2026-02-19T20:07:07.335 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-19T20:07:07.335 DEBUG:teuthology.orchestra.run.trial073:> sudo chmod 777 /var/log/ceph 2026-02-19T20:07:07.337 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod 777 /var/log/ceph 2026-02-19T20:07:07.360 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-19T20:07:07.360 DEBUG:teuthology.orchestra.run.trial073:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-19T20:07:07.399 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-19T20:07:07.423 INFO:tasks.ceph:Creating extra log directories... 2026-02-19T20:07:07.423 DEBUG:teuthology.orchestra.run.trial073:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-19T20:07:07.464 DEBUG:teuthology.orchestra.run.trial080:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-19T20:07:07.496 INFO:tasks.ceph:Creating ceph cluster cluster1... 2026-02-19T20:07:07.496 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-19T20:07:07.496 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606', 'branch': 'wip-yuri-testing-2026-02-18-1820', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'flavor': 'default', 'job_id': '59606', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2026-02-18-1820'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2026-02-18-1820', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}}, 'owner': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'seed': 9494, 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-yuri-testing-2026-02-18-1820', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_802bf111cd55e57f5842153d7ba2b8b964aa818a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a', 'targets': {'trial073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB5jildMU4Gxv7Me0NWGejXuXKJU0ATaIZa5AK7cNxK30rrYQN2V8cPhtJ5LLP4cLUwEh81LB7fQXNqP3T3bdrk=', 'trial080.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPW/JGWKJQToz3T5QPFuk5f7NULZNw/y2U6EqcR6aAdECz9epQf3o9nkHkkiTXJf2p8O4P+PHtQzkoVvKSwiJTI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}}, {'ceph': {'cluster': 'cluster2'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_MODE': 'snapshot', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-19_15:47:56', 'tube': 'trial', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-19T20:07:07.497 DEBUG:teuthology.orchestra.run.trial073:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2026-02-19T20:07:07.527 DEBUG:teuthology.orchestra.run.trial080:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2026-02-19T20:07:07.543 DEBUG:teuthology.orchestra.run.trial073:> sudo install -d -m0777 -- /var/run/ceph 2026-02-19T20:07:07.584 DEBUG:teuthology.orchestra.run.trial080:> sudo install -d -m0777 -- /var/run/ceph 2026-02-19T20:07:07.610 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:07.610 DEBUG:teuthology.orchestra.run.trial073:> dd if=/scratch_devs of=/dev/stdout 2026-02-19T20:07:07.662 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-19T20:07:07.662 DEBUG:teuthology.orchestra.run.trial073:> stat /dev/vg_nvme/lv_1 2026-02-19T20:07:07.718 INFO:teuthology.orchestra.run.trial073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-19T20:07:07.719 INFO:teuthology.orchestra.run.trial073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-19T20:07:07.719 INFO:teuthology.orchestra.run.trial073.stdout:Device: 6h/6d Inode: 1313 Links: 1 2026-02-19T20:07:07.719 INFO:teuthology.orchestra.run.trial073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-19T20:07:07.719 INFO:teuthology.orchestra.run.trial073.stdout:Context: system_u:object_r:device_t:s0 2026-02-19T20:07:07.719 INFO:teuthology.orchestra.run.trial073.stdout:Access: 2026-02-19 20:06:42.914375995 +0000 2026-02-19T20:07:07.719 INFO:teuthology.orchestra.run.trial073.stdout:Modify: 2026-02-19 20:05:56.967399753 +0000 2026-02-19T20:07:07.719 INFO:teuthology.orchestra.run.trial073.stdout:Change: 2026-02-19 20:05:56.967399753 +0000 2026-02-19T20:07:07.719 INFO:teuthology.orchestra.run.trial073.stdout: Birth: 2026-02-19 20:05:56.967399753 +0000 2026-02-19T20:07:07.720 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-19T20:07:07.780 INFO:teuthology.orchestra.run.trial073.stderr:1+0 records in 2026-02-19T20:07:07.780 INFO:teuthology.orchestra.run.trial073.stderr:1+0 records out 2026-02-19T20:07:07.781 INFO:teuthology.orchestra.run.trial073.stderr:512 bytes copied, 0.000158196 s, 3.2 MB/s 2026-02-19T20:07:07.782 DEBUG:teuthology.orchestra.run.trial073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-19T20:07:07.840 DEBUG:teuthology.orchestra.run.trial073:> stat /dev/vg_nvme/lv_2 2026-02-19T20:07:07.896 INFO:teuthology.orchestra.run.trial073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-19T20:07:07.896 INFO:teuthology.orchestra.run.trial073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-19T20:07:07.896 INFO:teuthology.orchestra.run.trial073.stdout:Device: 6h/6d Inode: 1322 Links: 1 2026-02-19T20:07:07.896 INFO:teuthology.orchestra.run.trial073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-19T20:07:07.896 INFO:teuthology.orchestra.run.trial073.stdout:Context: system_u:object_r:device_t:s0 2026-02-19T20:07:07.896 INFO:teuthology.orchestra.run.trial073.stdout:Access: 2026-02-19 20:06:42.914375995 +0000 2026-02-19T20:07:07.897 INFO:teuthology.orchestra.run.trial073.stdout:Modify: 2026-02-19 20:05:56.967399753 +0000 2026-02-19T20:07:07.897 INFO:teuthology.orchestra.run.trial073.stdout:Change: 2026-02-19 20:05:56.967399753 +0000 2026-02-19T20:07:07.897 INFO:teuthology.orchestra.run.trial073.stdout: Birth: 2026-02-19 20:05:56.967399753 +0000 2026-02-19T20:07:07.897 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-19T20:07:07.961 INFO:teuthology.orchestra.run.trial073.stderr:1+0 records in 2026-02-19T20:07:07.962 INFO:teuthology.orchestra.run.trial073.stderr:1+0 records out 2026-02-19T20:07:07.962 INFO:teuthology.orchestra.run.trial073.stderr:512 bytes copied, 0.000156914 s, 3.3 MB/s 2026-02-19T20:07:07.963 DEBUG:teuthology.orchestra.run.trial073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-19T20:07:08.020 DEBUG:teuthology.orchestra.run.trial073:> stat /dev/vg_nvme/lv_3 2026-02-19T20:07:08.076 INFO:teuthology.orchestra.run.trial073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-19T20:07:08.077 INFO:teuthology.orchestra.run.trial073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-19T20:07:08.077 INFO:teuthology.orchestra.run.trial073.stdout:Device: 6h/6d Inode: 1298 Links: 1 2026-02-19T20:07:08.077 INFO:teuthology.orchestra.run.trial073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-19T20:07:08.077 INFO:teuthology.orchestra.run.trial073.stdout:Context: system_u:object_r:device_t:s0 2026-02-19T20:07:08.077 INFO:teuthology.orchestra.run.trial073.stdout:Access: 2026-02-19 20:06:42.914375995 +0000 2026-02-19T20:07:08.077 INFO:teuthology.orchestra.run.trial073.stdout:Modify: 2026-02-19 20:05:56.965399798 +0000 2026-02-19T20:07:08.077 INFO:teuthology.orchestra.run.trial073.stdout:Change: 2026-02-19 20:05:56.965399798 +0000 2026-02-19T20:07:08.077 INFO:teuthology.orchestra.run.trial073.stdout: Birth: 2026-02-19 20:05:56.965399798 +0000 2026-02-19T20:07:08.078 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-19T20:07:08.138 INFO:teuthology.orchestra.run.trial073.stderr:1+0 records in 2026-02-19T20:07:08.138 INFO:teuthology.orchestra.run.trial073.stderr:1+0 records out 2026-02-19T20:07:08.138 INFO:teuthology.orchestra.run.trial073.stderr:512 bytes copied, 0.000219691 s, 2.3 MB/s 2026-02-19T20:07:08.139 DEBUG:teuthology.orchestra.run.trial073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-19T20:07:08.196 DEBUG:teuthology.orchestra.run.trial073:> stat /dev/vg_nvme/lv_4 2026-02-19T20:07:08.253 INFO:teuthology.orchestra.run.trial073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-19T20:07:08.253 INFO:teuthology.orchestra.run.trial073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-19T20:07:08.254 INFO:teuthology.orchestra.run.trial073.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-19T20:07:08.254 INFO:teuthology.orchestra.run.trial073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-19T20:07:08.254 INFO:teuthology.orchestra.run.trial073.stdout:Context: system_u:object_r:device_t:s0 2026-02-19T20:07:08.254 INFO:teuthology.orchestra.run.trial073.stdout:Access: 2026-02-19 20:06:42.915375974 +0000 2026-02-19T20:07:08.254 INFO:teuthology.orchestra.run.trial073.stdout:Modify: 2026-02-19 20:05:56.966399776 +0000 2026-02-19T20:07:08.254 INFO:teuthology.orchestra.run.trial073.stdout:Change: 2026-02-19 20:05:56.966399776 +0000 2026-02-19T20:07:08.254 INFO:teuthology.orchestra.run.trial073.stdout: Birth: 2026-02-19 20:05:56.966399776 +0000 2026-02-19T20:07:08.254 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-19T20:07:08.317 INFO:teuthology.orchestra.run.trial073.stderr:1+0 records in 2026-02-19T20:07:08.317 INFO:teuthology.orchestra.run.trial073.stderr:1+0 records out 2026-02-19T20:07:08.317 INFO:teuthology.orchestra.run.trial073.stderr:512 bytes copied, 0.000208159 s, 2.5 MB/s 2026-02-19T20:07:08.318 DEBUG:teuthology.orchestra.run.trial073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-19T20:07:08.376 INFO:tasks.ceph:osd dev map: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2026-02-19T20:07:08.376 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial073.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-19T20:07:08.376 INFO:tasks.ceph:Generating config... 2026-02-19T20:07:08.378 INFO:tasks.ceph:[client] debug rbd = 20 2026-02-19T20:07:08.378 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2026-02-19T20:07:08.378 INFO:tasks.ceph:[client] log to stderr = False 2026-02-19T20:07:08.378 INFO:tasks.ceph:[client] rbd default features = 125 2026-02-19T20:07:08.378 INFO:tasks.ceph:[client] rbd mirror image policy type = simple 2026-02-19T20:07:08.378 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.379 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.379 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.379 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.379 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.379 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.379 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.379 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.379 INFO:tasks.ceph:[global] mon client directed command retry = 5 2026-02-19T20:07:08.379 INFO:tasks.ceph:[global] mon warn on pool no app = False 2026-02-19T20:07:08.379 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2026-02-19T20:07:08.379 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-19T20:07:08.379 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-19T20:07:08.379 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-19T20:07:08.379 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-19T20:07:08.379 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-19T20:07:08.380 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-19T20:07:08.381 INFO:tasks.ceph:Setting up cluster1.mon.a... 2026-02-19T20:07:08.381 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster1.keyring 2026-02-19T20:07:08.448 INFO:teuthology.orchestra.run.trial073.stdout:creating /etc/ceph/cluster1.keyring 2026-02-19T20:07:08.451 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster1.keyring 2026-02-19T20:07:08.531 DEBUG:teuthology.orchestra.run.trial073:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2026-02-19T20:07:08.603 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster1.mon.a', '10.20.193.73')] 2026-02-19T20:07:08.604 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.73', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2026-02-19T20:07:08.604 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:08.605 DEBUG:teuthology.orchestra.run.trial073:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-19T20:07:08.661 DEBUG:teuthology.orchestra.run.trial073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.73 --print /home/ubuntu/cephtest/cluster1.monmap 2026-02-19T20:07:08.730 INFO:teuthology.orchestra.run.trial073.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster1.monmap 2026-02-19T20:07:08.730 INFO:teuthology.orchestra.run.trial073.stdout:monmaptool: generated fsid a74de42f-1722-4818-ae18-8cf060b80e9c 2026-02-19T20:07:08.730 INFO:teuthology.orchestra.run.trial073.stdout:setting min_mon_release = tentacle 2026-02-19T20:07:08.730 INFO:teuthology.orchestra.run.trial073.stdout:epoch 0 2026-02-19T20:07:08.730 INFO:teuthology.orchestra.run.trial073.stdout:fsid a74de42f-1722-4818-ae18-8cf060b80e9c 2026-02-19T20:07:08.730 INFO:teuthology.orchestra.run.trial073.stdout:last_changed 2026-02-19T20:07:08.728925+0000 2026-02-19T20:07:08.730 INFO:teuthology.orchestra.run.trial073.stdout:created 2026-02-19T20:07:08.728925+0000 2026-02-19T20:07:08.730 INFO:teuthology.orchestra.run.trial073.stdout:min_mon_release 20 (tentacle) 2026-02-19T20:07:08.730 INFO:teuthology.orchestra.run.trial073.stdout:election_strategy: 1 2026-02-19T20:07:08.731 INFO:teuthology.orchestra.run.trial073.stdout:0: [v2:10.20.193.73:3300/0,v1:10.20.193.73:6789/0] mon.a 2026-02-19T20:07:08.731 INFO:teuthology.orchestra.run.trial073.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster1.monmap (1 monitors) 2026-02-19T20:07:08.731 DEBUG:teuthology.orchestra.run.trial073:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-19T20:07:08.787 INFO:tasks.ceph:Writing /etc/ceph/cluster1.conf for FSID a74de42f-1722-4818-ae18-8cf060b80e9c... 2026-02-19T20:07:08.787 DEBUG:teuthology.orchestra.run.trial073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2026-02-19T20:07:08.828 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2026-02-19T20:07:08.869 INFO:teuthology.orchestra.run.trial073.stdout:[global] 2026-02-19T20:07:08.869 INFO:teuthology.orchestra.run.trial073.stdout: chdir = "" 2026-02-19T20:07:08.869 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-19T20:07:08.869 INFO:teuthology.orchestra.run.trial073.stdout: auth supported = cephx 2026-02-19T20:07:08.869 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: filestore xattr use omap = true 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: mon clock drift allowed = 1.000 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: osd crush chooseleaf type = 0 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: auth debug = true 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: ms die on old message = true 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: ms die on bug = true 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: mon pg warn max object skew = 0 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: # disable pg_autoscaler by default for new pools 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: osd pool default size = 2 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: mon osd allow primary affinity = true 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: mon osd allow pg remap = true 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on legacy crush tunables = false 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on crush straw calc version zero = false 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on no sortbitwise = false 2026-02-19T20:07:08.870 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on osd down out interval zero = false 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on too few osds = false 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon_allow_pool_size_one = true 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: osd default data pool replay window = 5 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon allow pool delete = true 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon cluster log file level = debug 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: debug asserts on shutdown = true 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon health detail to clog = false 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon host = 10.20.193.73 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon client directed command retry = 5 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on pool no app = False 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: ms inject socket failures = 5000 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: fsid = a74de42f-1722-4818-ae18-8cf060b80e9c 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout:[osd] 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: osd journal size = 100 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: osd scrub load threshold = 5.0 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: osd scrub max interval = 600 2026-02-19T20:07:08.871 INFO:teuthology.orchestra.run.trial073.stdout: osd mclock profile = high_recovery_ops 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd mclock skip benchmark = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd recover clone overlap = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd recovery max chunk = 1048576 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd debug shutdown = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd debug op order = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd debug verify stray on activate = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd debug trim objects = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd open classes on start = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd debug pg log writeout = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd deep scrub update digest min age = 30 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd map max advance = 10 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: journal zero on create = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: filestore ondisk finisher threads = 3 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: filestore apply finisher threads = 3 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: bdev debug aio = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: osd debug misdirected ops = true 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: bluestore block size = 96636764160 2026-02-19T20:07:08.872 INFO:teuthology.orchestra.run.trial073.stdout: bluestore compression algorithm = zlib 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: bluestore compression mode = aggressive 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: bluestore fsck on mount = True 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug bluefs = 1/20 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug bluestore = 1/20 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug ms = 1 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug osd = 20 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug rocksdb = 4/10 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: mon osd backfillfull_ratio = 0.85 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: mon osd full ratio = 0.9 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: mon osd nearfull ratio = 0.8 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: osd failsafe full ratio = 0.95 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: osd objectstore = bluestore 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout:[mgr] 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug ms = 1 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug mgr = 20 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug mon = 20 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug auth = 20 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: mon reweight min pgs per osd = 4 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: mon reweight min bytes per osd = 10 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: mgr/telemetry/nag = false 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout:[mon] 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug ms = 1 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug mon = 20 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug paxos = 20 2026-02-19T20:07:08.873 INFO:teuthology.orchestra.run.trial073.stdout: debug auth = 20 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon data avail warn = 5 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon mgr mkfs grace = 240 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon reweight min pgs per osd = 4 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon osd reporter subtree level = osd 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon osd prime pg temp = true 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon reweight min bytes per osd = 10 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: auth mon ticket ttl = 660 # 11m 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: auth service ticket ttl = 240 # 4m 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: # don't complain about insecure global_id in the test suite 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: # 1m isn't quite enough 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon_down_mkfs_grace = 2m 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_filestore_osds = false 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout:[client] 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: rgw cache enabled = true 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: rgw enable ops log = true 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: rgw enable usage log = true 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-19T20:07:08.874 INFO:teuthology.orchestra.run.trial073.stdout: debug rbd = 20 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: debug rbd_mirror = 15 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: log to stderr = False 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: rbd default features = 125 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: rbd mirror image policy type = simple 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout:[mon.a] 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.0] 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.1] 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.2] 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.3] 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial080.stdout:[global] 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial080.stdout: chdir = "" 2026-02-19T20:07:08.875 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: auth supported = cephx 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: filestore xattr use omap = true 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: mon clock drift allowed = 1.000 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: osd crush chooseleaf type = 0 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: auth debug = true 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: ms die on old message = true 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: ms die on bug = true 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: mon pg warn max object skew = 0 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: # disable pg_autoscaler by default for new pools 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: osd pool default size = 2 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: mon osd allow primary affinity = true 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: mon osd allow pg remap = true 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on legacy crush tunables = false 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on crush straw calc version zero = false 2026-02-19T20:07:08.876 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on no sortbitwise = false 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on osd down out interval zero = false 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on too few osds = false 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon_allow_pool_size_one = true 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: osd default data pool replay window = 5 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon allow pool delete = true 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon cluster log file level = debug 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: debug asserts on shutdown = true 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon health detail to clog = false 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon host = 10.20.193.73 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon client directed command retry = 5 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on pool no app = False 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: ms inject socket failures = 5000 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: fsid = a74de42f-1722-4818-ae18-8cf060b80e9c 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout:[osd] 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: osd journal size = 100 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: osd scrub load threshold = 5.0 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: osd scrub max interval = 600 2026-02-19T20:07:08.877 INFO:teuthology.orchestra.run.trial080.stdout: osd mclock profile = high_recovery_ops 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd mclock skip benchmark = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd recover clone overlap = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd recovery max chunk = 1048576 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd debug shutdown = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd debug op order = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd debug verify stray on activate = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd debug trim objects = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd open classes on start = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd debug pg log writeout = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd deep scrub update digest min age = 30 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd map max advance = 10 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: journal zero on create = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: filestore ondisk finisher threads = 3 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: filestore apply finisher threads = 3 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: bdev debug aio = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: osd debug misdirected ops = true 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: bluestore block size = 96636764160 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: bluestore compression algorithm = zlib 2026-02-19T20:07:08.878 INFO:teuthology.orchestra.run.trial080.stdout: bluestore compression mode = aggressive 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: bluestore fsck on mount = True 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug bluefs = 1/20 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug bluestore = 1/20 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug osd = 20 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug rocksdb = 4/10 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: mon osd backfillfull_ratio = 0.85 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: mon osd full ratio = 0.9 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: mon osd nearfull ratio = 0.8 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: osd failsafe full ratio = 0.95 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: osd objectstore = bluestore 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout:[mgr] 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug mgr = 20 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug mon = 20 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug auth = 20 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min pgs per osd = 4 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min bytes per osd = 10 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: mgr/telemetry/nag = false 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout:[mon] 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug mon = 20 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug paxos = 20 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: debug auth = 20 2026-02-19T20:07:08.879 INFO:teuthology.orchestra.run.trial080.stdout: mon data avail warn = 5 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: mon mgr mkfs grace = 240 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min pgs per osd = 4 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: mon osd reporter subtree level = osd 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: mon osd prime pg temp = true 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min bytes per osd = 10 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: auth mon ticket ttl = 660 # 11m 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: auth service ticket ttl = 240 # 4m 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: # don't complain about insecure global_id in the test suite 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: # 1m isn't quite enough 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: mon_down_mkfs_grace = 2m 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_filestore_osds = false 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout:[client] 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: rgw cache enabled = true 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: rgw enable ops log = true 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: rgw enable usage log = true 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-19T20:07:08.880 INFO:teuthology.orchestra.run.trial080.stdout: debug rbd = 20 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: debug rbd_mirror = 15 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: log to stderr = False 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: rbd default features = 125 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: rbd mirror image policy type = simple 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout:[mon.a] 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.0] 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.1] 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.2] 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.3] 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:08.881 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:08.882 INFO:tasks.ceph:Creating admin key on cluster1.mon.a... 2026-02-19T20:07:08.882 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster1.keyring 2026-02-19T20:07:08.962 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-19T20:07:08.962 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:08.962 DEBUG:teuthology.orchestra.run.trial073:> dd if=/etc/ceph/cluster1.keyring of=/dev/stdout 2026-02-19T20:07:09.018 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:09.018 DEBUG:teuthology.orchestra.run.trial073:> dd if=/home/ubuntu/cephtest/cluster1.monmap of=/dev/stdout 2026-02-19T20:07:09.073 INFO:tasks.ceph:Sending monmap to node ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:07:09.073 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:09.073 DEBUG:teuthology.orchestra.run.trial073:> sudo dd of=/etc/ceph/cluster1.keyring 2026-02-19T20:07:09.073 DEBUG:teuthology.orchestra.run.trial073:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2026-02-19T20:07:09.147 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:09.147 DEBUG:teuthology.orchestra.run.trial073:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2026-02-19T20:07:09.204 INFO:tasks.ceph:Sending monmap to node ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:07:09.204 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:09.204 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/etc/ceph/cluster1.keyring 2026-02-19T20:07:09.204 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2026-02-19T20:07:09.240 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:09.240 DEBUG:teuthology.orchestra.run.trial080:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2026-02-19T20:07:09.296 INFO:tasks.ceph:Setting up mon nodes... 2026-02-19T20:07:09.296 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-19T20:07:09.297 DEBUG:teuthology.orchestra.run.trial073:> sudo mkdir -p /var/lib/ceph/mgr/cluster1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster1-x/keyring 2026-02-19T20:07:09.339 INFO:teuthology.orchestra.run.trial073.stdout:creating /var/lib/ceph/mgr/cluster1-x/keyring 2026-02-19T20:07:09.341 INFO:tasks.ceph:Setting up mds nodes... 2026-02-19T20:07:09.341 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-19T20:07:09.341 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster1.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.0.keyring 2026-02-19T20:07:09.414 INFO:teuthology.orchestra.run.trial073.stdout:creating /etc/ceph/cluster1.client.0.keyring 2026-02-19T20:07:09.424 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster1.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.keyring 2026-02-19T20:07:09.459 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster1.client.mirror.keyring 2026-02-19T20:07:09.470 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster1.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.0.keyring 2026-02-19T20:07:09.543 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster1.client.mirror.0.keyring 2026-02-19T20:07:09.554 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster1.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.1.keyring 2026-02-19T20:07:09.627 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster1.client.mirror.1.keyring 2026-02-19T20:07:09.638 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster1.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.2.keyring 2026-02-19T20:07:09.712 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster1.client.mirror.2.keyring 2026-02-19T20:07:09.723 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster1.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.3.keyring 2026-02-19T20:07:09.795 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster1.client.mirror.3.keyring 2026-02-19T20:07:09.806 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster1.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.4.keyring 2026-02-19T20:07:09.879 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster1.client.mirror.4.keyring 2026-02-19T20:07:09.890 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster1.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.5.keyring 2026-02-19T20:07:09.962 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster1.client.mirror.5.keyring 2026-02-19T20:07:09.973 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster1.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.6.keyring 2026-02-19T20:07:10.045 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster1.client.mirror.6.keyring 2026-02-19T20:07:10.056 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-19T20:07:10.056 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial073.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-19T20:07:10.056 DEBUG:teuthology.orchestra.run.trial073:> sudo mkdir -p /var/lib/ceph/osd/cluster1-0 2026-02-19T20:07:10.081 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2026-02-19T20:07:10.082 INFO:tasks.ceph:role: cluster1.osd.0 2026-02-19T20:07:10.082 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:07:10.082 DEBUG:teuthology.orchestra.run.trial073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-19T20:07:10.147 INFO:teuthology.orchestra.run.trial073.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-19T20:07:10.148 INFO:teuthology.orchestra.run.trial073.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-19T20:07:10.148 INFO:teuthology.orchestra.run.trial073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-19T20:07:10.148 INFO:teuthology.orchestra.run.trial073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-19T20:07:10.148 INFO:teuthology.orchestra.run.trial073.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-19T20:07:10.148 INFO:teuthology.orchestra.run.trial073.stdout: = sunit=0 swidth=0 blks 2026-02-19T20:07:10.148 INFO:teuthology.orchestra.run.trial073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-19T20:07:10.148 INFO:teuthology.orchestra.run.trial073.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-19T20:07:10.148 INFO:teuthology.orchestra.run.trial073.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-19T20:07:10.149 INFO:teuthology.orchestra.run.trial073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-19T20:07:10.733 INFO:teuthology.orchestra.run.trial073.stdout:Discarding blocks...Done. 2026-02-19T20:07:10.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial073.front.sepia.ceph.com -o noatime 2026-02-19T20:07:10.735 DEBUG:teuthology.orchestra.run.trial073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster1-0 2026-02-19T20:07:10.804 DEBUG:teuthology.orchestra.run.trial073:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-0 2026-02-19T20:07:10.872 DEBUG:teuthology.orchestra.run.trial073:> sudo mkdir -p /var/lib/ceph/osd/cluster1-1 2026-02-19T20:07:10.935 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2026-02-19T20:07:10.936 INFO:tasks.ceph:role: cluster1.osd.1 2026-02-19T20:07:10.936 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:07:10.936 DEBUG:teuthology.orchestra.run.trial073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-19T20:07:11.000 INFO:teuthology.orchestra.run.trial073.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-19T20:07:11.001 INFO:teuthology.orchestra.run.trial073.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-19T20:07:11.001 INFO:teuthology.orchestra.run.trial073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-19T20:07:11.001 INFO:teuthology.orchestra.run.trial073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-19T20:07:11.001 INFO:teuthology.orchestra.run.trial073.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-19T20:07:11.001 INFO:teuthology.orchestra.run.trial073.stdout: = sunit=0 swidth=0 blks 2026-02-19T20:07:11.001 INFO:teuthology.orchestra.run.trial073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-19T20:07:11.001 INFO:teuthology.orchestra.run.trial073.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-19T20:07:11.001 INFO:teuthology.orchestra.run.trial073.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-19T20:07:11.001 INFO:teuthology.orchestra.run.trial073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-19T20:07:11.571 INFO:teuthology.orchestra.run.trial073.stdout:Discarding blocks...Done. 2026-02-19T20:07:11.573 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial073.front.sepia.ceph.com -o noatime 2026-02-19T20:07:11.573 DEBUG:teuthology.orchestra.run.trial073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster1-1 2026-02-19T20:07:11.642 DEBUG:teuthology.orchestra.run.trial073:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-1 2026-02-19T20:07:11.708 DEBUG:teuthology.orchestra.run.trial073:> sudo mkdir -p /var/lib/ceph/osd/cluster1-2 2026-02-19T20:07:11.771 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2026-02-19T20:07:11.771 INFO:tasks.ceph:role: cluster1.osd.2 2026-02-19T20:07:11.772 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:07:11.772 DEBUG:teuthology.orchestra.run.trial073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-19T20:07:11.837 INFO:teuthology.orchestra.run.trial073.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-19T20:07:11.837 INFO:teuthology.orchestra.run.trial073.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-19T20:07:11.838 INFO:teuthology.orchestra.run.trial073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-19T20:07:11.838 INFO:teuthology.orchestra.run.trial073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-19T20:07:11.838 INFO:teuthology.orchestra.run.trial073.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-19T20:07:11.838 INFO:teuthology.orchestra.run.trial073.stdout: = sunit=0 swidth=0 blks 2026-02-19T20:07:11.838 INFO:teuthology.orchestra.run.trial073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-19T20:07:11.838 INFO:teuthology.orchestra.run.trial073.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-19T20:07:11.838 INFO:teuthology.orchestra.run.trial073.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-19T20:07:11.838 INFO:teuthology.orchestra.run.trial073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-19T20:07:12.398 INFO:teuthology.orchestra.run.trial073.stdout:Discarding blocks...Done. 2026-02-19T20:07:12.400 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial073.front.sepia.ceph.com -o noatime 2026-02-19T20:07:12.400 DEBUG:teuthology.orchestra.run.trial073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster1-2 2026-02-19T20:07:12.468 DEBUG:teuthology.orchestra.run.trial073:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-2 2026-02-19T20:07:12.535 DEBUG:teuthology.orchestra.run.trial073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster1.monmap 2026-02-19T20:07:12.612 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:12.608+0000 7f6174c31780 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-0/keyring: can't open /var/lib/ceph/osd/cluster1-0/keyring: (2) No such file or directory 2026-02-19T20:07:12.613 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:12.608+0000 7f6174c31780 -1 created new key in keyring /var/lib/ceph/osd/cluster1-0/keyring 2026-02-19T20:07:12.613 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:12.609+0000 7f6174c31780 -1 bdev(0x2545400 /var/lib/ceph/osd/cluster1-0/block) open stat got: (1) Operation not permitted 2026-02-19T20:07:12.613 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:12.609+0000 7f6174c31780 -1 bluestore(/var/lib/ceph/osd/cluster1-0) _read_fsid unparsable uuid 2026-02-19T20:07:13.052 DEBUG:teuthology.orchestra.run.trial073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-0 2026-02-19T20:07:13.117 DEBUG:teuthology.orchestra.run.trial073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster1.monmap 2026-02-19T20:07:13.192 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:13.188+0000 7fa8ae230780 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-1/keyring: can't open /var/lib/ceph/osd/cluster1-1/keyring: (2) No such file or directory 2026-02-19T20:07:13.192 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:13.188+0000 7fa8ae230780 -1 created new key in keyring /var/lib/ceph/osd/cluster1-1/keyring 2026-02-19T20:07:13.192 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:13.188+0000 7fa8ae230780 -1 bdev(0x40bb400 /var/lib/ceph/osd/cluster1-1/block) open stat got: (1) Operation not permitted 2026-02-19T20:07:13.192 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:13.189+0000 7fa8ae230780 -1 bluestore(/var/lib/ceph/osd/cluster1-1) _read_fsid unparsable uuid 2026-02-19T20:07:13.653 DEBUG:teuthology.orchestra.run.trial073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-1 2026-02-19T20:07:13.718 DEBUG:teuthology.orchestra.run.trial073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster1.monmap 2026-02-19T20:07:13.795 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:13.792+0000 7f6af9c2d780 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-2/keyring: can't open /var/lib/ceph/osd/cluster1-2/keyring: (2) No such file or directory 2026-02-19T20:07:13.795 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:13.792+0000 7f6af9c2d780 -1 created new key in keyring /var/lib/ceph/osd/cluster1-2/keyring 2026-02-19T20:07:13.795 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:13.792+0000 7f6af9c2d780 -1 bdev(0x2e8d400 /var/lib/ceph/osd/cluster1-2/block) open stat got: (1) Operation not permitted 2026-02-19T20:07:13.795 INFO:teuthology.orchestra.run.trial073.stderr:2026-02-19T20:07:13.792+0000 7f6af9c2d780 -1 bluestore(/var/lib/ceph/osd/cluster1-2) _read_fsid unparsable uuid 2026-02-19T20:07:14.240 DEBUG:teuthology.orchestra.run.trial073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-2 2026-02-19T20:07:14.304 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-19T20:07:14.305 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:14.305 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/var/lib/ceph/mgr/cluster1-x/keyring of=/dev/stdout 2026-02-19T20:07:14.369 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:14.370 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/var/lib/ceph/osd/cluster1-0/keyring of=/dev/stdout 2026-02-19T20:07:14.434 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:14.434 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/var/lib/ceph/osd/cluster1-1/keyring of=/dev/stdout 2026-02-19T20:07:14.496 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:14.497 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/var/lib/ceph/osd/cluster1-2/keyring of=/dev/stdout 2026-02-19T20:07:14.561 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:14.561 DEBUG:teuthology.orchestra.run.trial073:> dd if=/etc/ceph/cluster1.client.0.keyring of=/dev/stdout 2026-02-19T20:07:14.618 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:14.618 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster1.client.mirror.keyring of=/dev/stdout 2026-02-19T20:07:14.637 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:14.637 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster1.client.mirror.0.keyring of=/dev/stdout 2026-02-19T20:07:14.694 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:14.694 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster1.client.mirror.1.keyring of=/dev/stdout 2026-02-19T20:07:14.750 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:14.750 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster1.client.mirror.2.keyring of=/dev/stdout 2026-02-19T20:07:14.806 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:14.806 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster1.client.mirror.3.keyring of=/dev/stdout 2026-02-19T20:07:14.864 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:14.864 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster1.client.mirror.4.keyring of=/dev/stdout 2026-02-19T20:07:14.919 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:14.919 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster1.client.mirror.5.keyring of=/dev/stdout 2026-02-19T20:07:14.975 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:14.975 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster1.client.mirror.6.keyring of=/dev/stdout 2026-02-19T20:07:15.031 INFO:tasks.ceph:Adding keys to all mons... 2026-02-19T20:07:15.031 DEBUG:teuthology.orchestra.run.trial073:> sudo tee -a /etc/ceph/cluster1.keyring 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[mgr.x] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBtbZdpJvMlFBAAQI4jcX1Z2jmcSrvgrd81Ew== 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[osd.0] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBwbZdpul5iJBAAjajZAF8P5S3HQbDlhvYBDQ== 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[osd.1] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBxbZdpIrRXCxAAgfT0xjTHVWSey0c/iEuTlg== 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[osd.2] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBxbZdp6ixULxAAic0Q8iBrwAyuG97tlc+dxA== 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[client.0] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBtbZdpwH+WGBAA+gCdcPD2s8KjzjC0Hjfi9g== 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBtbZdpUY1gGxAAAXlSnZ0YVolJQcugW7fcPg== 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.0] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBtbZdp4HhcIBAAUgpYwrAOLI9teLNvsPIukQ== 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.1] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBtbZdpa59aJRAA0jbVFxzazmDaWlHMpXI0Vw== 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.2] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBtbZdppRFzKhAA0f7hYs4eiKKPRJ+A0LxLGQ== 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.3] 2026-02-19T20:07:15.055 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBtbZdpvuhjLxAAuP/6RH3v3TGDHFAUKPgD/Q== 2026-02-19T20:07:15.056 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.4] 2026-02-19T20:07:15.056 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBtbZdpto5fNBAAj7RN1jAN3v6G+6m/hg04Sw== 2026-02-19T20:07:15.056 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.5] 2026-02-19T20:07:15.056 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBtbZdp/ZZXORAAP0oYtQ5wMBxxUfiFPhn8LQ== 2026-02-19T20:07:15.056 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.6] 2026-02-19T20:07:15.056 INFO:teuthology.orchestra.run.trial073.stdout: key = AQBubZdp9mqzAhAAThnpbZDTsrK0dLjQ4Zv/+w== 2026-02-19T20:07:15.057 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-19T20:07:15.137 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-19T20:07:15.214 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-19T20:07:15.290 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-19T20:07:15.367 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:15.447 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:15.527 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:15.606 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:15.686 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:15.766 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:15.846 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:15.925 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:16.004 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:16.082 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-19T20:07:16.082 DEBUG:teuthology.orchestra.run.trial073:> sudo mkdir -p /var/lib/ceph/mon/cluster1-a 2026-02-19T20:07:16.145 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster1 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster1.monmap --keyring /etc/ceph/cluster1.keyring 2026-02-19T20:07:16.230 DEBUG:teuthology.orchestra.run.trial073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster1-a 2026-02-19T20:07:16.255 DEBUG:teuthology.orchestra.run.trial073:> rm -- /home/ubuntu/cephtest/cluster1.monmap 2026-02-19T20:07:16.310 INFO:tasks.ceph:Starting mon daemons in cluster cluster1... 2026-02-19T20:07:16.311 INFO:tasks.ceph.cluster1.mon.a:Restarting daemon 2026-02-19T20:07:16.311 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster1 -i a 2026-02-19T20:07:16.352 INFO:tasks.ceph.cluster1.mon.a:Started 2026-02-19T20:07:16.353 INFO:tasks.ceph:Starting mgr daemons in cluster cluster1... 2026-02-19T20:07:16.353 INFO:tasks.ceph.cluster1.mgr.x:Restarting daemon 2026-02-19T20:07:16.353 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster1 -i x 2026-02-19T20:07:16.355 INFO:tasks.ceph.cluster1.mgr.x:Started 2026-02-19T20:07:16.356 DEBUG:tasks.ceph:set 0 configs 2026-02-19T20:07:16.356 DEBUG:teuthology.orchestra.run.trial073:> sudo ceph --cluster cluster1 config dump 2026-02-19T20:07:16.452 INFO:teuthology.orchestra.run.trial073.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-19T20:07:16.460 INFO:tasks.ceph:Setting crush tunables to default 2026-02-19T20:07:16.460 DEBUG:teuthology.orchestra.run.trial073:> sudo ceph --cluster cluster1 osd crush tunables default 2026-02-19T20:07:16.560 INFO:teuthology.orchestra.run.trial073.stderr:adjusted tunables profile to default 2026-02-19T20:07:16.568 INFO:tasks.ceph:check_enable_crimson: False 2026-02-19T20:07:16.568 INFO:tasks.ceph:Starting osd daemons in cluster cluster1... 2026-02-19T20:07:16.568 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:16.568 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/var/lib/ceph/osd/cluster1-0/fsid of=/dev/stdout 2026-02-19T20:07:16.593 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:16.593 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/var/lib/ceph/osd/cluster1-1/fsid of=/dev/stdout 2026-02-19T20:07:16.659 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:16.659 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/var/lib/ceph/osd/cluster1-2/fsid of=/dev/stdout 2026-02-19T20:07:16.723 DEBUG:teuthology.orchestra.run.trial073:> sudo ceph --cluster cluster1 osd new 0d1fcb52-009c-4059-9bbe-e781f759ca72 0 2026-02-19T20:07:16.866 INFO:teuthology.orchestra.run.trial073.stdout:0 2026-02-19T20:07:16.875 DEBUG:teuthology.orchestra.run.trial073:> sudo ceph --cluster cluster1 osd new c749e329-91bc-42fa-b289-58af874876ee 1 2026-02-19T20:07:16.978 INFO:teuthology.orchestra.run.trial073.stdout:1 2026-02-19T20:07:16.987 DEBUG:teuthology.orchestra.run.trial073:> sudo ceph --cluster cluster1 osd new 77997b69-c81d-4af7-adc0-c3d775ce9285 2 2026-02-19T20:07:17.088 INFO:teuthology.orchestra.run.trial073.stdout:2 2026-02-19T20:07:17.096 INFO:tasks.ceph.cluster1.osd.0:Restarting daemon 2026-02-19T20:07:17.097 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 0 2026-02-19T20:07:17.099 INFO:tasks.ceph.cluster1.osd.0:Started 2026-02-19T20:07:17.099 INFO:tasks.ceph.cluster1.osd.1:Restarting daemon 2026-02-19T20:07:17.099 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 1 2026-02-19T20:07:17.102 INFO:tasks.ceph.cluster1.osd.1:Started 2026-02-19T20:07:17.102 INFO:tasks.ceph.cluster1.osd.2:Restarting daemon 2026-02-19T20:07:17.102 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 2 2026-02-19T20:07:17.104 INFO:tasks.ceph.cluster1.osd.2:Started 2026-02-19T20:07:17.105 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2026-02-19T20:07:17.202 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:17.202 INFO:teuthology.orchestra.run.trial073.stdout:{"epoch":5,"fsid":"a74de42f-1722-4818-ae18-8cf060b80e9c","created":"2026-02-19T20:07:16.412396+0000","modified":"2026-02-19T20:07:17.086411+0000","last_up_change":"0.000000","last_in_change":"2026-02-19T20:07:17.086411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0d1fcb52-009c-4059-9bbe-e781f759ca72","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c749e329-91bc-42fa-b289-58af874876ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"77997b69-c81d-4af7-adc0-c3d775ce9285","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:17.212 INFO:tasks.ceph.ceph_manager.cluster1:[] 2026-02-19T20:07:17.212 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-19T20:07:17.325 INFO:tasks.ceph.cluster1.osd.2.trial073.stderr:2026-02-19T20:07:17.321+0000 7fe2e6068780 -1 Falling back to public interface 2026-02-19T20:07:17.330 INFO:tasks.ceph.cluster1.osd.1.trial073.stderr:2026-02-19T20:07:17.326+0000 7f84c125e780 -1 Falling back to public interface 2026-02-19T20:07:17.330 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:07:17.326+0000 7f636bdd8780 -1 Falling back to public interface 2026-02-19T20:07:17.331 INFO:tasks.ceph.cluster1.osd.1.trial073.stderr:2026-02-19T20:07:17.326+0000 7f84bf1fd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2026-02-19T20:07:17.460 INFO:tasks.ceph.cluster1.mgr.x.trial073.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-19T20:07:17.461 INFO:tasks.ceph.cluster1.mgr.x.trial073.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-19T20:07:17.461 INFO:tasks.ceph.cluster1.mgr.x.trial073.stderr: from numpy import show_config as show_numpy_config 2026-02-19T20:07:17.514 DEBUG:teuthology.orchestra.run.trial073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2026-02-19T20:07:17.819 INFO:teuthology.misc.health.trial073.stdout: 2026-02-19T20:07:17.819 INFO:teuthology.misc.health.trial073.stdout:{"epoch":5,"fsid":"a74de42f-1722-4818-ae18-8cf060b80e9c","created":"2026-02-19T20:07:16.412396+0000","modified":"2026-02-19T20:07:17.086411+0000","last_up_change":"0.000000","last_in_change":"2026-02-19T20:07:17.086411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0d1fcb52-009c-4059-9bbe-e781f759ca72","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c749e329-91bc-42fa-b289-58af874876ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"77997b69-c81d-4af7-adc0-c3d775ce9285","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:17.834 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-19T20:07:17.988 INFO:tasks.ceph.cluster1.osd.2.trial073.stderr:2026-02-19T20:07:17.985+0000 7fe2e6068780 -1 osd.2 0 log_to_monitors true 2026-02-19T20:07:18.001 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:07:17.998+0000 7f636bdd8780 -1 osd.0 0 log_to_monitors true 2026-02-19T20:07:18.006 INFO:tasks.ceph.cluster1.osd.1.trial073.stderr:2026-02-19T20:07:18.003+0000 7f84c125e780 -1 osd.1 0 log_to_monitors true 2026-02-19T20:07:19.422 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:07:19.418+0000 7f63673f7640 -1 osd.0 0 waiting for initial osdmap 2026-02-19T20:07:19.423 INFO:tasks.ceph.cluster1.osd.2.trial073.stderr:2026-02-19T20:07:19.419+0000 7fe2e0ff7640 -1 osd.2 0 waiting for initial osdmap 2026-02-19T20:07:19.423 INFO:tasks.ceph.cluster1.osd.1.trial073.stderr:2026-02-19T20:07:19.419+0000 7f84bc1f7640 -1 osd.1 0 waiting for initial osdmap 2026-02-19T20:07:19.425 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:07:19.421+0000 7f635f32a640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-19T20:07:19.425 INFO:tasks.ceph.cluster1.osd.2.trial073.stderr:2026-02-19T20:07:19.422+0000 7fe2d8fe7640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-19T20:07:19.425 INFO:tasks.ceph.cluster1.osd.1.trial073.stderr:2026-02-19T20:07:19.422+0000 7f84b41e7640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-19T20:07:19.857 INFO:tasks.ceph.cluster1.mgr.x.trial073.stderr:2026-02-19T20:07:19.854+0000 7f694af6a640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-19T20:07:24.136 DEBUG:teuthology.orchestra.run.trial073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2026-02-19T20:07:24.306 INFO:teuthology.misc.health.trial073.stdout: 2026-02-19T20:07:24.306 INFO:teuthology.misc.health.trial073.stdout:{"epoch":11,"fsid":"a74de42f-1722-4818-ae18-8cf060b80e9c","created":"2026-02-19T20:07:16.412396+0000","modified":"2026-02-19T20:07:23.857289+0000","last_up_change":"2026-02-19T20:07:20.419166+0000","last_in_change":"2026-02-19T20:07:17.086411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-19T20:07:20.864703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0d1fcb52-009c-4059-9bbe-e781f759ca72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6808","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6810","nonce":358751968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6812","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6814","nonce":358751968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6819","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6820","nonce":358751968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6815","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6817","nonce":358751968}]},"public_addr":"10.20.193.73:6810/358751968","cluster_addr":"10.20.193.73:6814/358751968","heartbeat_back_addr":"10.20.193.73:6820/358751968","heartbeat_front_addr":"10.20.193.73:6817/358751968","state":["exists","up"]},{"osd":1,"uuid":"c749e329-91bc-42fa-b289-58af874876ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6809","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6811","nonce":3087991977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6813","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6816","nonce":3087991977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6822","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6823","nonce":3087991977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6818","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6821","nonce":3087991977}]},"public_addr":"10.20.193.73:6811/3087991977","cluster_addr":"10.20.193.73:6816/3087991977","heartbeat_back_addr":"10.20.193.73:6823/3087991977","heartbeat_front_addr":"10.20.193.73:6821/3087991977","state":["exists","up"]},{"osd":2,"uuid":"77997b69-c81d-4af7-adc0-c3d775ce9285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6800","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6801","nonce":211675687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6802","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6803","nonce":211675687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6806","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6807","nonce":211675687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6804","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6805","nonce":211675687}]},"public_addr":"10.20.193.73:6801/211675687","cluster_addr":"10.20.193.73:6803/211675687","heartbeat_back_addr":"10.20.193.73:6807/211675687","heartbeat_front_addr":"10.20.193.73:6805/211675687","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:19.035906+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:19.055463+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:18.967627+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:24.313 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-19T20:07:24.313 INFO:tasks.ceph:Creating RBD pool 2026-02-19T20:07:24.313 DEBUG:teuthology.orchestra.run.trial073:> sudo ceph --cluster cluster1 osd pool create rbd 8 2026-02-19T20:07:24.863 INFO:teuthology.orchestra.run.trial073.stderr:pool 'rbd' created 2026-02-19T20:07:24.873 DEBUG:teuthology.orchestra.run.trial073:> rbd --cluster cluster1 pool init rbd 2026-02-19T20:07:27.877 INFO:tasks.ceph:Starting mds daemons in cluster cluster1... 2026-02-19T20:07:27.878 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 config log 1 --format=json 2026-02-19T20:07:27.879 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-19T20:07:28.058 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:28.067 INFO:teuthology.orchestra.run.trial073.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-19T20:07:28.067 INFO:tasks.ceph_manager:config epoch is 1 2026-02-19T20:07:28.067 INFO:tasks.ceph:Waiting until cluster1 daemons up and pgs clean... 2026-02-19T20:07:28.068 INFO:tasks.ceph.ceph_manager.cluster1:waiting for mgr available 2026-02-19T20:07:28.068 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 mgr dump --format=json 2026-02-19T20:07:28.278 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:28.285 INFO:teuthology.orchestra.run.trial073.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6824","nonce":3827160224},{"type":"v1","addr":"10.20.193.73:6825","nonce":3827160224}]},"active_addr":"10.20.193.73:6825/3827160224","active_change":"2026-02-19T20:07:18.849917+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.73:0","nonce":3742207211}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.73:0","nonce":2412481240}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.73:0","nonce":61047241}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.73:0","nonce":528929861}]}]} 2026-02-19T20:07:28.290 INFO:tasks.ceph.ceph_manager.cluster1:mgr available! 2026-02-19T20:07:28.290 INFO:tasks.ceph.ceph_manager.cluster1:waiting for all up 2026-02-19T20:07:28.290 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2026-02-19T20:07:28.463 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:28.464 INFO:teuthology.orchestra.run.trial073.stdout:{"epoch":15,"fsid":"a74de42f-1722-4818-ae18-8cf060b80e9c","created":"2026-02-19T20:07:16.412396+0000","modified":"2026-02-19T20:07:27.866600+0000","last_up_change":"2026-02-19T20:07:20.419166+0000","last_in_change":"2026-02-19T20:07:17.086411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-19T20:07:20.864703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-19T20:07:24.487762+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0d1fcb52-009c-4059-9bbe-e781f759ca72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6808","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6810","nonce":358751968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6812","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6814","nonce":358751968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6819","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6820","nonce":358751968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6815","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6817","nonce":358751968}]},"public_addr":"10.20.193.73:6810/358751968","cluster_addr":"10.20.193.73:6814/358751968","heartbeat_back_addr":"10.20.193.73:6820/358751968","heartbeat_front_addr":"10.20.193.73:6817/358751968","state":["exists","up"]},{"osd":1,"uuid":"c749e329-91bc-42fa-b289-58af874876ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6809","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6811","nonce":3087991977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6813","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6816","nonce":3087991977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6822","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6823","nonce":3087991977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6818","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6821","nonce":3087991977}]},"public_addr":"10.20.193.73:6811/3087991977","cluster_addr":"10.20.193.73:6816/3087991977","heartbeat_back_addr":"10.20.193.73:6823/3087991977","heartbeat_front_addr":"10.20.193.73:6821/3087991977","state":["exists","up"]},{"osd":2,"uuid":"77997b69-c81d-4af7-adc0-c3d775ce9285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6800","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6801","nonce":211675687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6802","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6803","nonce":211675687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6806","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6807","nonce":211675687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6804","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6805","nonce":211675687}]},"public_addr":"10.20.193.73:6801/211675687","cluster_addr":"10.20.193.73:6803/211675687","heartbeat_back_addr":"10.20.193.73:6807/211675687","heartbeat_front_addr":"10.20.193.73:6805/211675687","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:19.035906+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:19.055463+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:18.967627+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:28.472 INFO:tasks.ceph.ceph_manager.cluster1:all up! 2026-02-19T20:07:28.472 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2026-02-19T20:07:28.646 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:28.646 INFO:teuthology.orchestra.run.trial073.stdout:{"epoch":15,"fsid":"a74de42f-1722-4818-ae18-8cf060b80e9c","created":"2026-02-19T20:07:16.412396+0000","modified":"2026-02-19T20:07:27.866600+0000","last_up_change":"2026-02-19T20:07:20.419166+0000","last_in_change":"2026-02-19T20:07:17.086411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-19T20:07:20.864703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-19T20:07:24.487762+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0d1fcb52-009c-4059-9bbe-e781f759ca72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6808","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6810","nonce":358751968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6812","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6814","nonce":358751968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6819","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6820","nonce":358751968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6815","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6817","nonce":358751968}]},"public_addr":"10.20.193.73:6810/358751968","cluster_addr":"10.20.193.73:6814/358751968","heartbeat_back_addr":"10.20.193.73:6820/358751968","heartbeat_front_addr":"10.20.193.73:6817/358751968","state":["exists","up"]},{"osd":1,"uuid":"c749e329-91bc-42fa-b289-58af874876ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6809","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6811","nonce":3087991977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6813","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6816","nonce":3087991977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6822","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6823","nonce":3087991977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6818","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6821","nonce":3087991977}]},"public_addr":"10.20.193.73:6811/3087991977","cluster_addr":"10.20.193.73:6816/3087991977","heartbeat_back_addr":"10.20.193.73:6823/3087991977","heartbeat_front_addr":"10.20.193.73:6821/3087991977","state":["exists","up"]},{"osd":2,"uuid":"77997b69-c81d-4af7-adc0-c3d775ce9285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6800","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6801","nonce":211675687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6802","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6803","nonce":211675687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6806","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6807","nonce":211675687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6804","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6805","nonce":211675687}]},"public_addr":"10.20.193.73:6801/211675687","cluster_addr":"10.20.193.73:6803/211675687","heartbeat_back_addr":"10.20.193.73:6807/211675687","heartbeat_front_addr":"10.20.193.73:6805/211675687","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:19.035906+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:19.055463+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:18.967627+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:28.655 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 flush_pg_stats 2026-02-19T20:07:28.656 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 flush_pg_stats 2026-02-19T20:07:28.656 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 flush_pg_stats 2026-02-19T20:07:28.742 INFO:teuthology.orchestra.run.trial073.stdout:34359738372 2026-02-19T20:07:28.743 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2026-02-19T20:07:28.744 INFO:teuthology.orchestra.run.trial073.stdout:34359738371 2026-02-19T20:07:28.745 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2026-02-19T20:07:28.745 INFO:teuthology.orchestra.run.trial073.stdout:34359738371 2026-02-19T20:07:28.746 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2026-02-19T20:07:28.965 INFO:teuthology.orchestra.run.trial073.stdout:34359738371 2026-02-19T20:07:28.966 INFO:teuthology.orchestra.run.trial073.stdout:34359738372 2026-02-19T20:07:28.966 INFO:teuthology.orchestra.run.trial073.stdout:34359738371 2026-02-19T20:07:28.976 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738371 for osd.0 2026-02-19T20:07:28.976 DEBUG:teuthology.parallel:result is None 2026-02-19T20:07:28.976 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738372 got 34359738372 for osd.1 2026-02-19T20:07:28.977 DEBUG:teuthology.parallel:result is None 2026-02-19T20:07:28.977 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738371 for osd.2 2026-02-19T20:07:28.977 DEBUG:teuthology.parallel:result is None 2026-02-19T20:07:28.977 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2026-02-19T20:07:28.977 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2026-02-19T20:07:29.160 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:29.161 INFO:teuthology.orchestra.run.trial073.stderr:dumped all 2026-02-19T20:07:29.168 INFO:teuthology.orchestra.run.trial073.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-19T20:07:28.854442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81376,"kb_used_data":496,"kb_used_omap":23,"kb_used_meta":80808,"kb_avail":283034144,"statfs":{"total":289910292480,"available":289826963456,"internally_reserved":0,"allocated":507904,"data_stored":1292257,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":24375,"internal_metadata":82747593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991761"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.870713+0000","last_change":"2026-02-19T20:07:27.870930+0000","last_active":"2026-02-19T20:07:27.870713+0000","last_peered":"2026-02-19T20:07:27.870713+0000","last_clean":"2026-02-19T20:07:27.870713+0000","last_became_active":"2026-02-19T20:07:25.866966+0000","last_became_peered":"2026-02-19T20:07:25.866966+0000","last_unstale":"2026-02-19T20:07:27.870713+0000","last_undegraded":"2026-02-19T20:07:27.870713+0000","last_fullsized":"2026-02-19T20:07:27.870713+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:08:33.157778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053370000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.871048+0000","last_change":"2026-02-19T20:07:27.871242+0000","last_active":"2026-02-19T20:07:27.871048+0000","last_peered":"2026-02-19T20:07:27.871048+0000","last_clean":"2026-02-19T20:07:27.871048+0000","last_became_active":"2026-02-19T20:07:25.867278+0000","last_became_peered":"2026-02-19T20:07:25.867278+0000","last_unstale":"2026-02-19T20:07:27.871048+0000","last_undegraded":"2026-02-19T20:07:27.871048+0000","last_fullsized":"2026-02-19T20:07:27.871048+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T05:18:57.062974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081141899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.871250+0000","last_change":"2026-02-19T20:07:27.871411+0000","last_active":"2026-02-19T20:07:27.871250+0000","last_peered":"2026-02-19T20:07:27.871250+0000","last_clean":"2026-02-19T20:07:27.871250+0000","last_became_active":"2026-02-19T20:07:25.867250+0000","last_became_peered":"2026-02-19T20:07:25.867250+0000","last_unstale":"2026-02-19T20:07:27.871250+0000","last_undegraded":"2026-02-19T20:07:27.871250+0000","last_fullsized":"2026-02-19T20:07:27.871250+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:55:29.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094229399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.870988+0000","last_change":"2026-02-19T20:07:27.871199+0000","last_active":"2026-02-19T20:07:27.870988+0000","last_peered":"2026-02-19T20:07:27.870988+0000","last_clean":"2026-02-19T20:07:27.870988+0000","last_became_active":"2026-02-19T20:07:25.867041+0000","last_became_peered":"2026-02-19T20:07:25.867041+0000","last_unstale":"2026-02-19T20:07:27.870988+0000","last_undegraded":"2026-02-19T20:07:27.870988+0000","last_fullsized":"2026-02-19T20:07:27.870988+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:09:11.855713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079016000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.869613+0000","last_change":"2026-02-19T20:07:27.869613+0000","last_active":"2026-02-19T20:07:27.869613+0000","last_peered":"2026-02-19T20:07:27.869613+0000","last_clean":"2026-02-19T20:07:27.869613+0000","last_became_active":"2026-02-19T20:07:25.866844+0000","last_became_peered":"2026-02-19T20:07:25.866844+0000","last_unstale":"2026-02-19T20:07:27.869613+0000","last_undegraded":"2026-02-19T20:07:27.869613+0000","last_fullsized":"2026-02-19T20:07:27.869613+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:23.866049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055244400000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.985976+0000","last_change":"2026-02-19T20:07:27.986078+0000","last_active":"2026-02-19T20:07:27.985976+0000","last_peered":"2026-02-19T20:07:27.985976+0000","last_clean":"2026-02-19T20:07:27.985976+0000","last_became_active":"2026-02-19T20:07:25.867756+0000","last_became_peered":"2026-02-19T20:07:25.867756+0000","last_unstale":"2026-02-19T20:07:27.985976+0000","last_undegraded":"2026-02-19T20:07:27.985976+0000","last_fullsized":"2026-02-19T20:07:27.985976+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:22:24.470302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000248385,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.986014+0000","last_change":"2026-02-19T20:07:27.986095+0000","last_active":"2026-02-19T20:07:27.986014+0000","last_peered":"2026-02-19T20:07:27.986014+0000","last_clean":"2026-02-19T20:07:27.986014+0000","last_became_active":"2026-02-19T20:07:25.867520+0000","last_became_peered":"2026-02-19T20:07:25.867520+0000","last_unstale":"2026-02-19T20:07:27.986014+0000","last_undegraded":"2026-02-19T20:07:27.986014+0000","last_fullsized":"2026-02-19T20:07:27.986014+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:16:42.961584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026667900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.870779+0000","last_change":"2026-02-19T20:07:27.870999+0000","last_active":"2026-02-19T20:07:27.870779+0000","last_peered":"2026-02-19T20:07:27.870779+0000","last_clean":"2026-02-19T20:07:27.870779+0000","last_became_active":"2026-02-19T20:07:25.866443+0000","last_became_peered":"2026-02-19T20:07:25.866443+0000","last_unstale":"2026-02-19T20:07:27.870779+0000","last_undegraded":"2026-02-19T20:07:27.870779+0000","last_fullsized":"2026-02-19T20:07:27.870779+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:57:49.025933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059055499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'72","reported_seq":111,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.870210+0000","last_change":"2026-02-19T20:07:23.081147+0000","last_active":"2026-02-19T20:07:27.870210+0000","last_peered":"2026-02-19T20:07:27.870210+0000","last_clean":"2026-02-19T20:07:27.870210+0000","last_became_active":"2026-02-19T20:07:23.079045+0000","last_became_peered":"2026-02-19T20:07:23.079045+0000","last_unstale":"2026-02-19T20:07:27.870210+0000","last_undegraded":"2026-02-19T20:07:27.870210+0000","last_fullsized":"2026-02-19T20:07:27.870210+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_clean_scrub_stamp":"2026-02-19T20:07:21.856480+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T01:22:07.001571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":188,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":192512,"data_stored":628736,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":188,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":192512,"data_stored":628736,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:07:29.170 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2026-02-19T20:07:29.347 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:29.347 INFO:teuthology.orchestra.run.trial073.stderr:dumped all 2026-02-19T20:07:29.355 INFO:teuthology.orchestra.run.trial073.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-19T20:07:28.854442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81376,"kb_used_data":496,"kb_used_omap":23,"kb_used_meta":80808,"kb_avail":283034144,"statfs":{"total":289910292480,"available":289826963456,"internally_reserved":0,"allocated":507904,"data_stored":1292257,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":24375,"internal_metadata":82747593},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991761"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.870713+0000","last_change":"2026-02-19T20:07:27.870930+0000","last_active":"2026-02-19T20:07:27.870713+0000","last_peered":"2026-02-19T20:07:27.870713+0000","last_clean":"2026-02-19T20:07:27.870713+0000","last_became_active":"2026-02-19T20:07:25.866966+0000","last_became_peered":"2026-02-19T20:07:25.866966+0000","last_unstale":"2026-02-19T20:07:27.870713+0000","last_undegraded":"2026-02-19T20:07:27.870713+0000","last_fullsized":"2026-02-19T20:07:27.870713+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:08:33.157778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053370000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.871048+0000","last_change":"2026-02-19T20:07:27.871242+0000","last_active":"2026-02-19T20:07:27.871048+0000","last_peered":"2026-02-19T20:07:27.871048+0000","last_clean":"2026-02-19T20:07:27.871048+0000","last_became_active":"2026-02-19T20:07:25.867278+0000","last_became_peered":"2026-02-19T20:07:25.867278+0000","last_unstale":"2026-02-19T20:07:27.871048+0000","last_undegraded":"2026-02-19T20:07:27.871048+0000","last_fullsized":"2026-02-19T20:07:27.871048+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T05:18:57.062974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081141899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.871250+0000","last_change":"2026-02-19T20:07:27.871411+0000","last_active":"2026-02-19T20:07:27.871250+0000","last_peered":"2026-02-19T20:07:27.871250+0000","last_clean":"2026-02-19T20:07:27.871250+0000","last_became_active":"2026-02-19T20:07:25.867250+0000","last_became_peered":"2026-02-19T20:07:25.867250+0000","last_unstale":"2026-02-19T20:07:27.871250+0000","last_undegraded":"2026-02-19T20:07:27.871250+0000","last_fullsized":"2026-02-19T20:07:27.871250+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:55:29.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094229399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.870988+0000","last_change":"2026-02-19T20:07:27.871199+0000","last_active":"2026-02-19T20:07:27.870988+0000","last_peered":"2026-02-19T20:07:27.870988+0000","last_clean":"2026-02-19T20:07:27.870988+0000","last_became_active":"2026-02-19T20:07:25.867041+0000","last_became_peered":"2026-02-19T20:07:25.867041+0000","last_unstale":"2026-02-19T20:07:27.870988+0000","last_undegraded":"2026-02-19T20:07:27.870988+0000","last_fullsized":"2026-02-19T20:07:27.870988+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:09:11.855713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079016000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.869613+0000","last_change":"2026-02-19T20:07:27.869613+0000","last_active":"2026-02-19T20:07:27.869613+0000","last_peered":"2026-02-19T20:07:27.869613+0000","last_clean":"2026-02-19T20:07:27.869613+0000","last_became_active":"2026-02-19T20:07:25.866844+0000","last_became_peered":"2026-02-19T20:07:25.866844+0000","last_unstale":"2026-02-19T20:07:27.869613+0000","last_undegraded":"2026-02-19T20:07:27.869613+0000","last_fullsized":"2026-02-19T20:07:27.869613+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:23.866049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055244400000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.985976+0000","last_change":"2026-02-19T20:07:27.986078+0000","last_active":"2026-02-19T20:07:27.985976+0000","last_peered":"2026-02-19T20:07:27.985976+0000","last_clean":"2026-02-19T20:07:27.985976+0000","last_became_active":"2026-02-19T20:07:25.867756+0000","last_became_peered":"2026-02-19T20:07:25.867756+0000","last_unstale":"2026-02-19T20:07:27.985976+0000","last_undegraded":"2026-02-19T20:07:27.985976+0000","last_fullsized":"2026-02-19T20:07:27.985976+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:22:24.470302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000248385,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.986014+0000","last_change":"2026-02-19T20:07:27.986095+0000","last_active":"2026-02-19T20:07:27.986014+0000","last_peered":"2026-02-19T20:07:27.986014+0000","last_clean":"2026-02-19T20:07:27.986014+0000","last_became_active":"2026-02-19T20:07:25.867520+0000","last_became_peered":"2026-02-19T20:07:25.867520+0000","last_unstale":"2026-02-19T20:07:27.986014+0000","last_undegraded":"2026-02-19T20:07:27.986014+0000","last_fullsized":"2026-02-19T20:07:27.986014+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:16:42.961584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026667900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.870779+0000","last_change":"2026-02-19T20:07:27.870999+0000","last_active":"2026-02-19T20:07:27.870779+0000","last_peered":"2026-02-19T20:07:27.870779+0000","last_clean":"2026-02-19T20:07:27.870779+0000","last_became_active":"2026-02-19T20:07:25.866443+0000","last_became_peered":"2026-02-19T20:07:25.866443+0000","last_unstale":"2026-02-19T20:07:27.870779+0000","last_undegraded":"2026-02-19T20:07:27.870779+0000","last_fullsized":"2026-02-19T20:07:27.870779+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:57:49.025933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059055499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'72","reported_seq":111,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-19T20:07:27.870210+0000","last_change":"2026-02-19T20:07:23.081147+0000","last_active":"2026-02-19T20:07:27.870210+0000","last_peered":"2026-02-19T20:07:27.870210+0000","last_clean":"2026-02-19T20:07:27.870210+0000","last_became_active":"2026-02-19T20:07:23.079045+0000","last_became_peered":"2026-02-19T20:07:23.079045+0000","last_unstale":"2026-02-19T20:07:27.870210+0000","last_undegraded":"2026-02-19T20:07:27.870210+0000","last_fullsized":"2026-02-19T20:07:27.870210+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_clean_scrub_stamp":"2026-02-19T20:07:21.856480+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T01:22:07.001571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27080,"kb_used_data":120,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":94344760,"statfs":{"total":96636764160,"available":96609034240,"internally_reserved":0,"allocated":122880,"data_stored":34785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":188,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":192512,"data_stored":628736,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":188,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":192512,"data_stored":628736,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:07:29.357 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2026-02-19T20:07:29.357 INFO:tasks.ceph:Waiting until ceph cluster cluster1 is healthy... 2026-02-19T20:07:29.357 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy 2026-02-19T20:07:29.357 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 health --format=json 2026-02-19T20:07:29.558 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:29.558 INFO:teuthology.orchestra.run.trial073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-19T20:07:29.566 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy done 2026-02-19T20:07:29.566 INFO:teuthology.run_tasks:Running task ceph... 2026-02-19T20:07:29.622 INFO:tasks.ceph:Creating ceph cluster cluster2... 2026-02-19T20:07:29.622 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-19T20:07:29.623 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606', 'branch': 'wip-yuri-testing-2026-02-18-1820', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'flavor': 'default', 'job_id': '59606', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2026-02-18-1820'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2026-02-18-1820', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}}, 'owner': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'seed': 9494, 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-yuri-testing-2026-02-18-1820', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_802bf111cd55e57f5842153d7ba2b8b964aa818a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a', 'targets': {'trial073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB5jildMU4Gxv7Me0NWGejXuXKJU0ATaIZa5AK7cNxK30rrYQN2V8cPhtJ5LLP4cLUwEh81LB7fQXNqP3T3bdrk=', 'trial080.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPW/JGWKJQToz3T5QPFuk5f7NULZNw/y2U6EqcR6aAdECz9epQf3o9nkHkkiTXJf2p8O4P+PHtQzkoVvKSwiJTI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'flavor': 'default', 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}}, {'ceph': {'cluster': 'cluster2', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '802bf111cd55e57f5842153d7ba2b8b964aa818a'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_MODE': 'snapshot', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-19_15:47:56', 'tube': 'trial', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-19T20:07:29.624 DEBUG:teuthology.orchestra.run.trial073:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2026-02-19T20:07:29.626 DEBUG:teuthology.orchestra.run.trial080:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2026-02-19T20:07:29.644 DEBUG:teuthology.orchestra.run.trial073:> sudo install -d -m0777 -- /var/run/ceph 2026-02-19T20:07:29.683 DEBUG:teuthology.orchestra.run.trial080:> sudo install -d -m0777 -- /var/run/ceph 2026-02-19T20:07:29.709 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:29.709 DEBUG:teuthology.orchestra.run.trial080:> dd if=/scratch_devs of=/dev/stdout 2026-02-19T20:07:29.766 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-19T20:07:29.766 DEBUG:teuthology.orchestra.run.trial080:> stat /dev/vg_nvme/lv_1 2026-02-19T20:07:29.824 INFO:teuthology.orchestra.run.trial080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-19T20:07:29.824 INFO:teuthology.orchestra.run.trial080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-19T20:07:29.825 INFO:teuthology.orchestra.run.trial080.stdout:Device: 6h/6d Inode: 1301 Links: 1 2026-02-19T20:07:29.825 INFO:teuthology.orchestra.run.trial080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-19T20:07:29.825 INFO:teuthology.orchestra.run.trial080.stdout:Context: system_u:object_r:device_t:s0 2026-02-19T20:07:29.825 INFO:teuthology.orchestra.run.trial080.stdout:Access: 2026-02-19 20:07:04.855345491 +0000 2026-02-19T20:07:29.825 INFO:teuthology.orchestra.run.trial080.stdout:Modify: 2026-02-19 20:06:22.232908937 +0000 2026-02-19T20:07:29.825 INFO:teuthology.orchestra.run.trial080.stdout:Change: 2026-02-19 20:06:22.232908937 +0000 2026-02-19T20:07:29.825 INFO:teuthology.orchestra.run.trial080.stdout: Birth: 2026-02-19 20:06:22.232908937 +0000 2026-02-19T20:07:29.825 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-19T20:07:29.888 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records in 2026-02-19T20:07:29.888 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records out 2026-02-19T20:07:29.888 INFO:teuthology.orchestra.run.trial080.stderr:512 bytes copied, 0.000162124 s, 3.2 MB/s 2026-02-19T20:07:29.889 DEBUG:teuthology.orchestra.run.trial080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-19T20:07:29.950 DEBUG:teuthology.orchestra.run.trial080:> stat /dev/vg_nvme/lv_2 2026-02-19T20:07:30.007 INFO:teuthology.orchestra.run.trial080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-19T20:07:30.007 INFO:teuthology.orchestra.run.trial080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-19T20:07:30.007 INFO:teuthology.orchestra.run.trial080.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-19T20:07:30.007 INFO:teuthology.orchestra.run.trial080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-19T20:07:30.007 INFO:teuthology.orchestra.run.trial080.stdout:Context: system_u:object_r:device_t:s0 2026-02-19T20:07:30.007 INFO:teuthology.orchestra.run.trial080.stdout:Access: 2026-02-19 20:07:04.855345491 +0000 2026-02-19T20:07:30.007 INFO:teuthology.orchestra.run.trial080.stdout:Modify: 2026-02-19 20:06:22.232908937 +0000 2026-02-19T20:07:30.007 INFO:teuthology.orchestra.run.trial080.stdout:Change: 2026-02-19 20:06:22.232908937 +0000 2026-02-19T20:07:30.008 INFO:teuthology.orchestra.run.trial080.stdout: Birth: 2026-02-19 20:06:22.232908937 +0000 2026-02-19T20:07:30.008 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-19T20:07:30.069 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records in 2026-02-19T20:07:30.070 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records out 2026-02-19T20:07:30.070 INFO:teuthology.orchestra.run.trial080.stderr:512 bytes copied, 0.000151484 s, 3.4 MB/s 2026-02-19T20:07:30.071 DEBUG:teuthology.orchestra.run.trial080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-19T20:07:30.128 DEBUG:teuthology.orchestra.run.trial080:> stat /dev/vg_nvme/lv_3 2026-02-19T20:07:30.184 INFO:teuthology.orchestra.run.trial080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-19T20:07:30.184 INFO:teuthology.orchestra.run.trial080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-19T20:07:30.184 INFO:teuthology.orchestra.run.trial080.stdout:Device: 6h/6d Inode: 1306 Links: 1 2026-02-19T20:07:30.185 INFO:teuthology.orchestra.run.trial080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-19T20:07:30.185 INFO:teuthology.orchestra.run.trial080.stdout:Context: system_u:object_r:device_t:s0 2026-02-19T20:07:30.185 INFO:teuthology.orchestra.run.trial080.stdout:Access: 2026-02-19 20:07:04.855345491 +0000 2026-02-19T20:07:30.185 INFO:teuthology.orchestra.run.trial080.stdout:Modify: 2026-02-19 20:06:22.232908937 +0000 2026-02-19T20:07:30.185 INFO:teuthology.orchestra.run.trial080.stdout:Change: 2026-02-19 20:06:22.232908937 +0000 2026-02-19T20:07:30.185 INFO:teuthology.orchestra.run.trial080.stdout: Birth: 2026-02-19 20:06:22.232908937 +0000 2026-02-19T20:07:30.185 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-19T20:07:30.248 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records in 2026-02-19T20:07:30.248 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records out 2026-02-19T20:07:30.249 INFO:teuthology.orchestra.run.trial080.stderr:512 bytes copied, 0.000158167 s, 3.2 MB/s 2026-02-19T20:07:30.250 DEBUG:teuthology.orchestra.run.trial080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-19T20:07:30.307 DEBUG:teuthology.orchestra.run.trial080:> stat /dev/vg_nvme/lv_4 2026-02-19T20:07:30.363 INFO:teuthology.orchestra.run.trial080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-19T20:07:30.364 INFO:teuthology.orchestra.run.trial080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-19T20:07:30.364 INFO:teuthology.orchestra.run.trial080.stdout:Device: 6h/6d Inode: 1296 Links: 1 2026-02-19T20:07:30.364 INFO:teuthology.orchestra.run.trial080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-19T20:07:30.364 INFO:teuthology.orchestra.run.trial080.stdout:Context: system_u:object_r:device_t:s0 2026-02-19T20:07:30.364 INFO:teuthology.orchestra.run.trial080.stdout:Access: 2026-02-19 20:07:04.855345491 +0000 2026-02-19T20:07:30.364 INFO:teuthology.orchestra.run.trial080.stdout:Modify: 2026-02-19 20:06:22.231908944 +0000 2026-02-19T20:07:30.364 INFO:teuthology.orchestra.run.trial080.stdout:Change: 2026-02-19 20:06:22.231908944 +0000 2026-02-19T20:07:30.365 INFO:teuthology.orchestra.run.trial080.stdout: Birth: 2026-02-19 20:06:22.231908944 +0000 2026-02-19T20:07:30.365 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-19T20:07:30.427 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records in 2026-02-19T20:07:30.427 INFO:teuthology.orchestra.run.trial080.stderr:1+0 records out 2026-02-19T20:07:30.427 INFO:teuthology.orchestra.run.trial080.stderr:512 bytes copied, 0.000146003 s, 3.5 MB/s 2026-02-19T20:07:30.428 DEBUG:teuthology.orchestra.run.trial080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-19T20:07:30.486 INFO:tasks.ceph:osd dev map: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2026-02-19T20:07:30.486 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial080.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-19T20:07:30.487 INFO:tasks.ceph:Generating config... 2026-02-19T20:07:30.489 INFO:tasks.ceph:[client] debug rbd = 20 2026-02-19T20:07:30.489 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2026-02-19T20:07:30.489 INFO:tasks.ceph:[client] log to stderr = False 2026-02-19T20:07:30.489 INFO:tasks.ceph:[client] rbd default features = 125 2026-02-19T20:07:30.489 INFO:tasks.ceph:[client] rbd mirror image policy type = simple 2026-02-19T20:07:30.489 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.489 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.489 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.489 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.490 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.490 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.490 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.490 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.490 INFO:tasks.ceph:[global] mon client directed command retry = 5 2026-02-19T20:07:30.490 INFO:tasks.ceph:[global] mon warn on pool no app = False 2026-02-19T20:07:30.490 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2026-02-19T20:07:30.490 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-19T20:07:30.490 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-19T20:07:30.490 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-19T20:07:30.490 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-19T20:07:30.490 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-19T20:07:30.491 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-19T20:07:30.492 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-19T20:07:30.492 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-19T20:07:30.492 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-19T20:07:30.492 INFO:tasks.ceph:Setting up cluster2.mon.a... 2026-02-19T20:07:30.492 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster2.keyring 2026-02-19T20:07:30.558 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster2.keyring 2026-02-19T20:07:30.561 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster2.keyring 2026-02-19T20:07:30.634 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2026-02-19T20:07:30.699 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster2.mon.a', '10.20.193.80')] 2026-02-19T20:07:30.700 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.80', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2026-02-19T20:07:30.701 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:30.701 DEBUG:teuthology.orchestra.run.trial080:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-19T20:07:30.756 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.80 --print /home/ubuntu/cephtest/cluster2.monmap 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster2.monmap 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:monmaptool: generated fsid 28c96ac4-2d59-44da-9699-cda71e572ddf 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:setting min_mon_release = tentacle 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:epoch 0 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:fsid 28c96ac4-2d59-44da-9699-cda71e572ddf 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:last_changed 2026-02-19T20:07:30.825088+0000 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:created 2026-02-19T20:07:30.825088+0000 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:min_mon_release 20 (tentacle) 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:election_strategy: 1 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:0: [v2:10.20.193.80:3300/0,v1:10.20.193.80:6789/0] mon.a 2026-02-19T20:07:30.825 INFO:teuthology.orchestra.run.trial080.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster2.monmap (1 monitors) 2026-02-19T20:07:30.826 DEBUG:teuthology.orchestra.run.trial080:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-19T20:07:30.883 INFO:tasks.ceph:Writing /etc/ceph/cluster2.conf for FSID 28c96ac4-2d59-44da-9699-cda71e572ddf... 2026-02-19T20:07:30.884 DEBUG:teuthology.orchestra.run.trial073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2026-02-19T20:07:30.887 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2026-02-19T20:07:30.927 INFO:teuthology.orchestra.run.trial073.stdout:[global] 2026-02-19T20:07:30.927 INFO:teuthology.orchestra.run.trial073.stdout: chdir = "" 2026-02-19T20:07:30.927 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-19T20:07:30.927 INFO:teuthology.orchestra.run.trial073.stdout: auth supported = cephx 2026-02-19T20:07:30.927 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.927 INFO:teuthology.orchestra.run.trial073.stdout: filestore xattr use omap = true 2026-02-19T20:07:30.927 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: mon clock drift allowed = 1.000 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: osd crush chooseleaf type = 0 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: auth debug = true 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: ms die on old message = true 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: ms die on bug = true 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-19T20:07:30.928 INFO:teuthology.orchestra.run.trial073.stdout: mon pg warn max object skew = 0 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: # disable pg_autoscaler by default for new pools 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: osd pool default size = 2 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: mon osd allow primary affinity = true 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: mon osd allow pg remap = true 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on legacy crush tunables = false 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on crush straw calc version zero = false 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on no sortbitwise = false 2026-02-19T20:07:30.929 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on osd down out interval zero = false 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on too few osds = false 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: mon_allow_pool_size_one = true 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: osd default data pool replay window = 5 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: mon allow pool delete = true 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.930 INFO:teuthology.orchestra.run.trial073.stdout: mon cluster log file level = debug 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: debug asserts on shutdown = true 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: mon health detail to clog = false 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: mon host = 10.20.193.80 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: mon client directed command retry = 5 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: mon warn on pool no app = False 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: ms inject socket failures = 5000 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: fsid = 28c96ac4-2d59-44da-9699-cda71e572ddf 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout:[osd] 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: osd journal size = 100 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: osd scrub load threshold = 5.0 2026-02-19T20:07:30.931 INFO:teuthology.orchestra.run.trial073.stdout: osd scrub max interval = 600 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: osd mclock profile = high_recovery_ops 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: osd mclock skip benchmark = true 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: osd recover clone overlap = true 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: osd recovery max chunk = 1048576 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: osd debug shutdown = true 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: osd debug op order = true 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: osd debug verify stray on activate = true 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: osd debug trim objects = true 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.932 INFO:teuthology.orchestra.run.trial073.stdout: osd open classes on start = true 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: osd debug pg log writeout = true 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: osd deep scrub update digest min age = 30 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: osd map max advance = 10 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: journal zero on create = true 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: filestore ondisk finisher threads = 3 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: filestore apply finisher threads = 3 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: bdev debug aio = true 2026-02-19T20:07:30.933 INFO:teuthology.orchestra.run.trial073.stdout: osd debug misdirected ops = true 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: bluestore block size = 96636764160 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: bluestore compression algorithm = zlib 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: bluestore compression mode = aggressive 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: bluestore fsck on mount = True 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: debug bluefs = 1/20 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: debug bluestore = 1/20 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: debug ms = 1 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: debug osd = 20 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: debug rocksdb = 4/10 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: mon osd backfillfull_ratio = 0.85 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: mon osd full ratio = 0.9 2026-02-19T20:07:30.934 INFO:teuthology.orchestra.run.trial073.stdout: mon osd nearfull ratio = 0.8 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: osd failsafe full ratio = 0.95 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: osd objectstore = bluestore 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout:[mgr] 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: debug ms = 1 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: debug mgr = 20 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: debug mon = 20 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: debug auth = 20 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: mon reweight min pgs per osd = 4 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: mon reweight min bytes per osd = 10 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: mgr/telemetry/nag = false 2026-02-19T20:07:30.935 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout:[mon] 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: debug ms = 1 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: debug mon = 20 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: debug paxos = 20 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: debug auth = 20 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: mon data avail warn = 5 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: mon mgr mkfs grace = 240 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: mon reweight min pgs per osd = 4 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: mon osd reporter subtree level = osd 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: mon osd prime pg temp = true 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: mon reweight min bytes per osd = 10 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.936 INFO:teuthology.orchestra.run.trial073.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: auth mon ticket ttl = 660 # 11m 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: auth service ticket ttl = 240 # 4m 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: # don't complain about insecure global_id in the test suite 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: # 1m isn't quite enough 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: mon_down_mkfs_grace = 2m 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: mon_warn_on_filestore_osds = false 2026-02-19T20:07:30.937 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout:[client] 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: rgw cache enabled = true 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: rgw enable ops log = true 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: rgw enable usage log = true 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: debug rbd = 20 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: debug rbd_mirror = 15 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: log to stderr = False 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: rbd default features = 125 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout: rbd mirror image policy type = simple 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout:[mon.a] 2026-02-19T20:07:30.938 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.0] 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.1] 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.2] 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout:[client.mirror.3] 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.939 INFO:teuthology.orchestra.run.trial073.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.966 INFO:teuthology.orchestra.run.trial080.stdout:[global] 2026-02-19T20:07:30.966 INFO:teuthology.orchestra.run.trial080.stdout: chdir = "" 2026-02-19T20:07:30.966 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: auth supported = cephx 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: filestore xattr use omap = true 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: mon clock drift allowed = 1.000 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: osd crush chooseleaf type = 0 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: auth debug = true 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.967 INFO:teuthology.orchestra.run.trial080.stdout: ms die on old message = true 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: ms die on bug = true 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: mon pg warn max object skew = 0 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: # disable pg_autoscaler by default for new pools 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: osd pool default size = 2 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.968 INFO:teuthology.orchestra.run.trial080.stdout: mon osd allow primary affinity = true 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: mon osd allow pg remap = true 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on legacy crush tunables = false 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on crush straw calc version zero = false 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on no sortbitwise = false 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on osd down out interval zero = false 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on too few osds = false 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: mon_allow_pool_size_one = true 2026-02-19T20:07:30.969 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: osd default data pool replay window = 5 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: mon allow pool delete = true 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: mon cluster log file level = debug 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: debug asserts on shutdown = true 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: mon health detail to clog = false 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: mon host = 10.20.193.80 2026-02-19T20:07:30.970 INFO:teuthology.orchestra.run.trial080.stdout: mon client directed command retry = 5 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: mon warn on pool no app = False 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: ms inject socket failures = 5000 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: fsid = 28c96ac4-2d59-44da-9699-cda71e572ddf 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout:[osd] 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: osd journal size = 100 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: osd scrub load threshold = 5.0 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: osd scrub max interval = 600 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: osd mclock profile = high_recovery_ops 2026-02-19T20:07:30.971 INFO:teuthology.orchestra.run.trial080.stdout: osd mclock skip benchmark = true 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: osd recover clone overlap = true 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: osd recovery max chunk = 1048576 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: osd debug shutdown = true 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: osd debug op order = true 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: osd debug verify stray on activate = true 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: osd debug trim objects = true 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.972 INFO:teuthology.orchestra.run.trial080.stdout: osd open classes on start = true 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: osd debug pg log writeout = true 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: osd deep scrub update digest min age = 30 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: osd map max advance = 10 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: journal zero on create = true 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: filestore ondisk finisher threads = 3 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: filestore apply finisher threads = 3 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: bdev debug aio = true 2026-02-19T20:07:30.973 INFO:teuthology.orchestra.run.trial080.stdout: osd debug misdirected ops = true 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: bluestore block size = 96636764160 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: bluestore compression algorithm = zlib 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: bluestore compression mode = aggressive 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: bluestore fsck on mount = True 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: debug bluefs = 1/20 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: debug bluestore = 1/20 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: debug osd = 20 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: debug rocksdb = 4/10 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: mon osd backfillfull_ratio = 0.85 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: mon osd full ratio = 0.9 2026-02-19T20:07:30.974 INFO:teuthology.orchestra.run.trial080.stdout: mon osd nearfull ratio = 0.8 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: osd failsafe full ratio = 0.95 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: osd objectstore = bluestore 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout:[mgr] 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: debug mgr = 20 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: debug mon = 20 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: debug auth = 20 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min pgs per osd = 4 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min bytes per osd = 10 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: mgr/telemetry/nag = false 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.975 INFO:teuthology.orchestra.run.trial080.stdout:[mon] 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: debug ms = 1 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: debug mon = 20 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: debug paxos = 20 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: debug auth = 20 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: mon data avail warn = 5 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: mon mgr mkfs grace = 240 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min pgs per osd = 4 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: mon osd reporter subtree level = osd 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: mon osd prime pg temp = true 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: mon reweight min bytes per osd = 10 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.976 INFO:teuthology.orchestra.run.trial080.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: auth mon ticket ttl = 660 # 11m 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: auth service ticket ttl = 240 # 4m 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: # don't complain about insecure global_id in the test suite 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: # 1m isn't quite enough 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: mon_down_mkfs_grace = 2m 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: mon_warn_on_filestore_osds = false 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:30.977 INFO:teuthology.orchestra.run.trial080.stdout:[client] 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: rgw cache enabled = true 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: rgw enable ops log = true 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: rgw enable usage log = true 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: debug rbd = 20 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: debug rbd_mirror = 15 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: log to stderr = False 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: rbd default features = 125 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout: rbd mirror image policy type = simple 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout:[mon.a] 2026-02-19T20:07:30.978 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.0] 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.1] 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.2] 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.3] 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2026-02-19T20:07:30.979 INFO:teuthology.orchestra.run.trial080.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2026-02-19T20:07:30.981 INFO:tasks.ceph:Creating admin key on cluster2.mon.a... 2026-02-19T20:07:30.981 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster2.keyring 2026-02-19T20:07:31.061 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-19T20:07:31.062 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:31.062 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster2.keyring of=/dev/stdout 2026-02-19T20:07:31.119 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:31.119 DEBUG:teuthology.orchestra.run.trial080:> dd if=/home/ubuntu/cephtest/cluster2.monmap of=/dev/stdout 2026-02-19T20:07:31.176 INFO:tasks.ceph:Sending monmap to node ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:07:31.176 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:31.176 DEBUG:teuthology.orchestra.run.trial073:> sudo dd of=/etc/ceph/cluster2.keyring 2026-02-19T20:07:31.176 DEBUG:teuthology.orchestra.run.trial073:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2026-02-19T20:07:31.209 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:31.210 DEBUG:teuthology.orchestra.run.trial073:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2026-02-19T20:07:31.266 INFO:tasks.ceph:Sending monmap to node ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:07:31.266 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:31.266 DEBUG:teuthology.orchestra.run.trial080:> sudo dd of=/etc/ceph/cluster2.keyring 2026-02-19T20:07:31.267 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2026-02-19T20:07:31.302 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:31.302 DEBUG:teuthology.orchestra.run.trial080:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2026-02-19T20:07:31.358 INFO:tasks.ceph:Setting up mon nodes... 2026-02-19T20:07:31.358 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-19T20:07:31.359 DEBUG:teuthology.orchestra.run.trial073:> sudo mkdir -p /var/lib/ceph/mgr/cluster2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster2-x/keyring 2026-02-19T20:07:31.403 INFO:teuthology.orchestra.run.trial073.stdout:creating /var/lib/ceph/mgr/cluster2-x/keyring 2026-02-19T20:07:31.406 INFO:tasks.ceph:Setting up mds nodes... 2026-02-19T20:07:31.406 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-19T20:07:31.406 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster2.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.0.keyring 2026-02-19T20:07:31.439 INFO:teuthology.orchestra.run.trial073.stdout:creating /etc/ceph/cluster2.client.0.keyring 2026-02-19T20:07:31.450 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster2.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.keyring 2026-02-19T20:07:31.485 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster2.client.mirror.keyring 2026-02-19T20:07:31.496 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster2.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.0.keyring 2026-02-19T20:07:31.569 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster2.client.mirror.0.keyring 2026-02-19T20:07:31.581 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster2.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.1.keyring 2026-02-19T20:07:31.653 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster2.client.mirror.1.keyring 2026-02-19T20:07:31.663 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster2.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.2.keyring 2026-02-19T20:07:31.735 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster2.client.mirror.2.keyring 2026-02-19T20:07:31.747 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster2.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.3.keyring 2026-02-19T20:07:31.820 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster2.client.mirror.3.keyring 2026-02-19T20:07:31.831 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster2.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.4.keyring 2026-02-19T20:07:31.903 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster2.client.mirror.4.keyring 2026-02-19T20:07:31.915 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster2.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.5.keyring 2026-02-19T20:07:31.988 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster2.client.mirror.5.keyring 2026-02-19T20:07:31.999 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster2.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.6.keyring 2026-02-19T20:07:32.072 INFO:teuthology.orchestra.run.trial080.stdout:creating /etc/ceph/cluster2.client.mirror.6.keyring 2026-02-19T20:07:32.083 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-19T20:07:32.084 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial073.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@trial080.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-19T20:07:32.084 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /var/lib/ceph/osd/cluster2-0 2026-02-19T20:07:32.147 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2026-02-19T20:07:32.148 INFO:tasks.ceph:role: cluster2.osd.0 2026-02-19T20:07:32.148 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:07:32.148 DEBUG:teuthology.orchestra.run.trial080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-19T20:07:32.212 INFO:teuthology.orchestra.run.trial080.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-19T20:07:32.212 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-19T20:07:32.213 INFO:teuthology.orchestra.run.trial080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-19T20:07:32.213 INFO:teuthology.orchestra.run.trial080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-19T20:07:32.213 INFO:teuthology.orchestra.run.trial080.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-19T20:07:32.213 INFO:teuthology.orchestra.run.trial080.stdout: = sunit=0 swidth=0 blks 2026-02-19T20:07:32.213 INFO:teuthology.orchestra.run.trial080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-19T20:07:32.213 INFO:teuthology.orchestra.run.trial080.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-19T20:07:32.213 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-19T20:07:32.213 INFO:teuthology.orchestra.run.trial080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-19T20:07:32.783 INFO:teuthology.orchestra.run.trial080.stdout:Discarding blocks...Done. 2026-02-19T20:07:32.785 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial080.front.sepia.ceph.com -o noatime 2026-02-19T20:07:32.785 DEBUG:teuthology.orchestra.run.trial080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster2-0 2026-02-19T20:07:32.857 DEBUG:teuthology.orchestra.run.trial080:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-0 2026-02-19T20:07:32.924 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /var/lib/ceph/osd/cluster2-1 2026-02-19T20:07:32.990 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2026-02-19T20:07:32.990 INFO:tasks.ceph:role: cluster2.osd.1 2026-02-19T20:07:32.990 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:07:32.990 DEBUG:teuthology.orchestra.run.trial080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-19T20:07:33.055 INFO:teuthology.orchestra.run.trial080.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-19T20:07:33.055 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-19T20:07:33.055 INFO:teuthology.orchestra.run.trial080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-19T20:07:33.055 INFO:teuthology.orchestra.run.trial080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-19T20:07:33.056 INFO:teuthology.orchestra.run.trial080.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-19T20:07:33.056 INFO:teuthology.orchestra.run.trial080.stdout: = sunit=0 swidth=0 blks 2026-02-19T20:07:33.056 INFO:teuthology.orchestra.run.trial080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-19T20:07:33.056 INFO:teuthology.orchestra.run.trial080.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-19T20:07:33.056 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-19T20:07:33.056 INFO:teuthology.orchestra.run.trial080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-19T20:07:33.615 INFO:teuthology.orchestra.run.trial080.stdout:Discarding blocks...Done. 2026-02-19T20:07:33.617 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial080.front.sepia.ceph.com -o noatime 2026-02-19T20:07:33.617 DEBUG:teuthology.orchestra.run.trial080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster2-1 2026-02-19T20:07:33.684 DEBUG:teuthology.orchestra.run.trial080:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-1 2026-02-19T20:07:33.750 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /var/lib/ceph/osd/cluster2-2 2026-02-19T20:07:33.815 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2026-02-19T20:07:33.816 INFO:tasks.ceph:role: cluster2.osd.2 2026-02-19T20:07:33.816 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:07:33.816 DEBUG:teuthology.orchestra.run.trial080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-19T20:07:33.881 INFO:teuthology.orchestra.run.trial080.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-19T20:07:33.881 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-19T20:07:33.881 INFO:teuthology.orchestra.run.trial080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-19T20:07:33.882 INFO:teuthology.orchestra.run.trial080.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-19T20:07:33.882 INFO:teuthology.orchestra.run.trial080.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-19T20:07:33.882 INFO:teuthology.orchestra.run.trial080.stdout: = sunit=0 swidth=0 blks 2026-02-19T20:07:33.882 INFO:teuthology.orchestra.run.trial080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-19T20:07:33.882 INFO:teuthology.orchestra.run.trial080.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-19T20:07:33.882 INFO:teuthology.orchestra.run.trial080.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-19T20:07:33.882 INFO:teuthology.orchestra.run.trial080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-19T20:07:34.438 INFO:teuthology.orchestra.run.trial080.stdout:Discarding blocks...Done. 2026-02-19T20:07:34.440 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial080.front.sepia.ceph.com -o noatime 2026-02-19T20:07:34.440 DEBUG:teuthology.orchestra.run.trial080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster2-2 2026-02-19T20:07:34.507 DEBUG:teuthology.orchestra.run.trial080:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-2 2026-02-19T20:07:34.575 DEBUG:teuthology.orchestra.run.trial080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster2.monmap 2026-02-19T20:07:34.651 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:34.650+0000 7f0fa624d780 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-0/keyring: can't open /var/lib/ceph/osd/cluster2-0/keyring: (2) No such file or directory 2026-02-19T20:07:34.652 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:34.651+0000 7f0fa624d780 -1 created new key in keyring /var/lib/ceph/osd/cluster2-0/keyring 2026-02-19T20:07:34.652 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:34.651+0000 7f0fa624d780 -1 bdev(0x26b3400 /var/lib/ceph/osd/cluster2-0/block) open stat got: (1) Operation not permitted 2026-02-19T20:07:34.652 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:34.651+0000 7f0fa624d780 -1 bluestore(/var/lib/ceph/osd/cluster2-0) _read_fsid unparsable uuid 2026-02-19T20:07:35.100 DEBUG:teuthology.orchestra.run.trial080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-0 2026-02-19T20:07:35.166 DEBUG:teuthology.orchestra.run.trial080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster2.monmap 2026-02-19T20:07:35.243 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:35.242+0000 7f447902f780 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-1/keyring: can't open /var/lib/ceph/osd/cluster2-1/keyring: (2) No such file or directory 2026-02-19T20:07:35.243 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:35.242+0000 7f447902f780 -1 created new key in keyring /var/lib/ceph/osd/cluster2-1/keyring 2026-02-19T20:07:35.243 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:35.242+0000 7f447902f780 -1 bdev(0x4007400 /var/lib/ceph/osd/cluster2-1/block) open stat got: (1) Operation not permitted 2026-02-19T20:07:35.244 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:35.242+0000 7f447902f780 -1 bluestore(/var/lib/ceph/osd/cluster2-1) _read_fsid unparsable uuid 2026-02-19T20:07:35.711 DEBUG:teuthology.orchestra.run.trial080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-1 2026-02-19T20:07:35.777 DEBUG:teuthology.orchestra.run.trial080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster2.monmap 2026-02-19T20:07:35.853 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:35.852+0000 7f6d0ca10780 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-2/keyring: can't open /var/lib/ceph/osd/cluster2-2/keyring: (2) No such file or directory 2026-02-19T20:07:35.854 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:35.852+0000 7f6d0ca10780 -1 created new key in keyring /var/lib/ceph/osd/cluster2-2/keyring 2026-02-19T20:07:35.854 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:35.852+0000 7f6d0ca10780 -1 bdev(0x3bfb400 /var/lib/ceph/osd/cluster2-2/block) open stat got: (1) Operation not permitted 2026-02-19T20:07:35.854 INFO:teuthology.orchestra.run.trial080.stderr:2026-02-19T20:07:35.852+0000 7f6d0ca10780 -1 bluestore(/var/lib/ceph/osd/cluster2-2) _read_fsid unparsable uuid 2026-02-19T20:07:36.309 DEBUG:teuthology.orchestra.run.trial080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-2 2026-02-19T20:07:36.374 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-19T20:07:36.375 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:36.375 DEBUG:teuthology.orchestra.run.trial073:> sudo dd if=/var/lib/ceph/mgr/cluster2-x/keyring of=/dev/stdout 2026-02-19T20:07:36.402 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.402 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/cluster2-0/keyring of=/dev/stdout 2026-02-19T20:07:36.437 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.438 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/cluster2-1/keyring of=/dev/stdout 2026-02-19T20:07:36.500 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.500 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/cluster2-2/keyring of=/dev/stdout 2026-02-19T20:07:36.564 DEBUG:teuthology.orchestra.run.trial073:> set -ex 2026-02-19T20:07:36.564 DEBUG:teuthology.orchestra.run.trial073:> dd if=/etc/ceph/cluster2.client.0.keyring of=/dev/stdout 2026-02-19T20:07:36.581 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.581 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster2.client.mirror.keyring of=/dev/stdout 2026-02-19T20:07:36.622 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.622 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster2.client.mirror.0.keyring of=/dev/stdout 2026-02-19T20:07:36.678 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.679 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster2.client.mirror.1.keyring of=/dev/stdout 2026-02-19T20:07:36.736 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.736 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster2.client.mirror.2.keyring of=/dev/stdout 2026-02-19T20:07:36.792 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.792 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster2.client.mirror.3.keyring of=/dev/stdout 2026-02-19T20:07:36.848 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.848 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster2.client.mirror.4.keyring of=/dev/stdout 2026-02-19T20:07:36.903 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.903 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster2.client.mirror.5.keyring of=/dev/stdout 2026-02-19T20:07:36.959 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:36.959 DEBUG:teuthology.orchestra.run.trial080:> dd if=/etc/ceph/cluster2.client.mirror.6.keyring of=/dev/stdout 2026-02-19T20:07:37.015 INFO:tasks.ceph:Adding keys to all mons... 2026-02-19T20:07:37.016 DEBUG:teuthology.orchestra.run.trial080:> sudo tee -a /etc/ceph/cluster2.keyring 2026-02-19T20:07:37.078 INFO:teuthology.orchestra.run.trial080.stdout:[mgr.x] 2026-02-19T20:07:37.078 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpsWr7FxAAnykt+sLHRu7gJ9/F29cYzA== 2026-02-19T20:07:37.078 INFO:teuthology.orchestra.run.trial080.stdout:[osd.0] 2026-02-19T20:07:37.078 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCGbZdpF7XXJhAASEG8zwpNpuc8GIaZ/ut+LQ== 2026-02-19T20:07:37.078 INFO:teuthology.orchestra.run.trial080.stdout:[osd.1] 2026-02-19T20:07:37.078 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCHbZdpxJB3DhAAEVHKfr8ufOW+WaNS/uFaJA== 2026-02-19T20:07:37.078 INFO:teuthology.orchestra.run.trial080.stdout:[osd.2] 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCHbZdpqT/ZMhAA3+chXoYt8HTOM/r8FmhobA== 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout:[client.0] 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpap4gGhAAHbShH5iKMJlK/DwgCPHB+g== 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror] 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpfOHlHBAAM9UFnLGWD3/uZXWtULyhfg== 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.0] 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpmXvuIRAAcqpb7qkvG/nxupuzkZEDtQ== 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.1] 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpJ4jsJhAArKraHfk1U2Gux6CW5H2Qrg== 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.2] 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdp85/TKxAAdDfJThAAvEFVc1zwEwe5gA== 2026-02-19T20:07:37.079 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.3] 2026-02-19T20:07:37.080 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpgKLmMBAA0x81c0AFYbOH9eIMOV+ftA== 2026-02-19T20:07:37.080 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.4] 2026-02-19T20:07:37.080 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpaRTWNRAA3qZfSeWaTYkzjRGnqlxM/g== 2026-02-19T20:07:37.080 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.5] 2026-02-19T20:07:37.080 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpLlfgOhAAb0qJlnwzuO7Va/GGGfGu/Q== 2026-02-19T20:07:37.080 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.6] 2026-02-19T20:07:37.080 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCEbZdpCDlGBBAA69BcYLGDN9GD6ubfLoASEw== 2026-02-19T20:07:37.081 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-19T20:07:37.160 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-19T20:07:37.238 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-19T20:07:37.316 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-19T20:07:37.395 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:37.474 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:37.554 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:37.634 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:37.714 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:37.793 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:37.872 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:37.952 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:38.032 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-19T20:07:38.113 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-19T20:07:38.114 DEBUG:teuthology.orchestra.run.trial080:> sudo mkdir -p /var/lib/ceph/mon/cluster2-a 2026-02-19T20:07:38.179 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster2 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster2.monmap --keyring /etc/ceph/cluster2.keyring 2026-02-19T20:07:38.260 DEBUG:teuthology.orchestra.run.trial080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster2-a 2026-02-19T20:07:38.325 DEBUG:teuthology.orchestra.run.trial080:> rm -- /home/ubuntu/cephtest/cluster2.monmap 2026-02-19T20:07:38.381 INFO:tasks.ceph:Starting mon daemons in cluster cluster2... 2026-02-19T20:07:38.382 INFO:tasks.ceph.cluster2.mon.a:Restarting daemon 2026-02-19T20:07:38.382 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster2 -i a 2026-02-19T20:07:38.423 INFO:tasks.ceph.cluster2.mon.a:Started 2026-02-19T20:07:38.424 INFO:tasks.ceph:Starting mgr daemons in cluster cluster2... 2026-02-19T20:07:38.424 INFO:tasks.ceph.cluster2.mgr.x:Restarting daemon 2026-02-19T20:07:38.424 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster2 -i x 2026-02-19T20:07:38.427 INFO:tasks.ceph.cluster2.mgr.x:Started 2026-02-19T20:07:38.428 DEBUG:tasks.ceph:set 0 configs 2026-02-19T20:07:38.428 DEBUG:teuthology.orchestra.run.trial080:> sudo ceph --cluster cluster2 config dump 2026-02-19T20:07:38.525 INFO:teuthology.orchestra.run.trial080.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-19T20:07:38.534 INFO:tasks.ceph:Setting crush tunables to default 2026-02-19T20:07:38.534 DEBUG:teuthology.orchestra.run.trial080:> sudo ceph --cluster cluster2 osd crush tunables default 2026-02-19T20:07:38.634 INFO:teuthology.orchestra.run.trial080.stderr:adjusted tunables profile to default 2026-02-19T20:07:38.642 INFO:tasks.ceph:check_enable_crimson: False 2026-02-19T20:07:38.642 INFO:tasks.ceph:Starting osd daemons in cluster cluster2... 2026-02-19T20:07:38.642 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:38.642 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/cluster2-0/fsid of=/dev/stdout 2026-02-19T20:07:38.667 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:38.667 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/cluster2-1/fsid of=/dev/stdout 2026-02-19T20:07:38.734 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:07:38.734 DEBUG:teuthology.orchestra.run.trial080:> sudo dd if=/var/lib/ceph/osd/cluster2-2/fsid of=/dev/stdout 2026-02-19T20:07:38.799 DEBUG:teuthology.orchestra.run.trial080:> sudo ceph --cluster cluster2 osd new e174f3ab-c860-44cb-b827-38110018fec2 0 2026-02-19T20:07:38.941 INFO:teuthology.orchestra.run.trial080.stdout:0 2026-02-19T20:07:38.948 DEBUG:teuthology.orchestra.run.trial080:> sudo ceph --cluster cluster2 osd new 6d926fb6-1591-422d-8ca1-58fb499c2772 1 2026-02-19T20:07:39.051 INFO:teuthology.orchestra.run.trial080.stdout:1 2026-02-19T20:07:39.058 DEBUG:teuthology.orchestra.run.trial080:> sudo ceph --cluster cluster2 osd new b259ad1f-166b-47e6-81f9-3e585f1bf53d 2 2026-02-19T20:07:39.161 INFO:teuthology.orchestra.run.trial080.stdout:2 2026-02-19T20:07:39.170 INFO:tasks.ceph.cluster2.osd.0:Restarting daemon 2026-02-19T20:07:39.170 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 0 2026-02-19T20:07:39.180 INFO:tasks.ceph.cluster2.osd.0:Started 2026-02-19T20:07:39.180 INFO:tasks.ceph.cluster2.osd.1:Restarting daemon 2026-02-19T20:07:39.180 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 1 2026-02-19T20:07:39.182 INFO:tasks.ceph.cluster2.mgr.x.trial073.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-19T20:07:39.182 INFO:tasks.ceph.cluster2.mgr.x.trial073.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-19T20:07:39.182 INFO:tasks.ceph.cluster2.mgr.x.trial073.stderr: from numpy import show_config as show_numpy_config 2026-02-19T20:07:39.190 INFO:tasks.ceph.cluster2.osd.1:Started 2026-02-19T20:07:39.190 INFO:tasks.ceph.cluster2.osd.2:Restarting daemon 2026-02-19T20:07:39.191 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 2 2026-02-19T20:07:39.203 INFO:tasks.ceph.cluster2.osd.2:Started 2026-02-19T20:07:39.204 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2026-02-19T20:07:39.314 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:39.314 INFO:teuthology.orchestra.run.trial080.stdout:{"epoch":5,"fsid":"28c96ac4-2d59-44da-9699-cda71e572ddf","created":"2026-02-19T20:07:38.484071+0000","modified":"2026-02-19T20:07:39.160813+0000","last_up_change":"0.000000","last_in_change":"2026-02-19T20:07:39.160813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e174f3ab-c860-44cb-b827-38110018fec2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6d926fb6-1591-422d-8ca1-58fb499c2772","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b259ad1f-166b-47e6-81f9-3e585f1bf53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:39.321 INFO:tasks.ceph.ceph_manager.cluster2:[] 2026-02-19T20:07:39.321 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-19T20:07:39.430 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:07:39.429+0000 7f5c0a63a780 -1 Falling back to public interface 2026-02-19T20:07:39.442 INFO:tasks.ceph.cluster2.osd.1.trial080.stderr:2026-02-19T20:07:39.441+0000 7f1aa5e34780 -1 Falling back to public interface 2026-02-19T20:07:39.443 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:07:39.441+0000 7fab2ebd4780 -1 Falling back to public interface 2026-02-19T20:07:39.623 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2026-02-19T20:07:39.883 INFO:teuthology.misc.health.trial080.stderr:2026-02-19T20:07:39.882+0000 7f387bfff640 0 --2- 10.20.193.80:0/3112443710 >> v2:10.20.193.80:3300/0 conn(0x7f3884095a80 0x7f3884159110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2026-02-19T20:07:39.916 INFO:teuthology.misc.health.trial080.stdout: 2026-02-19T20:07:39.916 INFO:teuthology.misc.health.trial080.stdout:{"epoch":5,"fsid":"28c96ac4-2d59-44da-9699-cda71e572ddf","created":"2026-02-19T20:07:38.484071+0000","modified":"2026-02-19T20:07:39.160813+0000","last_up_change":"0.000000","last_in_change":"2026-02-19T20:07:39.160813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e174f3ab-c860-44cb-b827-38110018fec2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6d926fb6-1591-422d-8ca1-58fb499c2772","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b259ad1f-166b-47e6-81f9-3e585f1bf53d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:39.923 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-19T20:07:39.958 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:07:39.957+0000 7fab2ebd4780 -1 osd.2 0 log_to_monitors true 2026-02-19T20:07:39.958 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:07:39.957+0000 7f5c0a63a780 -1 osd.0 0 log_to_monitors true 2026-02-19T20:07:39.966 INFO:tasks.ceph.cluster2.osd.1.trial080.stderr:2026-02-19T20:07:39.965+0000 7f1aa5e34780 -1 osd.1 0 log_to_monitors true 2026-02-19T20:07:41.372 INFO:tasks.ceph.cluster2.mgr.x.trial073.stderr:2026-02-19T20:07:41.369+0000 7fc0c985e640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-19T20:07:41.495 INFO:tasks.ceph.cluster2.osd.1.trial080.stderr:2026-02-19T20:07:41.494+0000 7f1aa0df7640 -1 osd.1 0 waiting for initial osdmap 2026-02-19T20:07:41.496 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:07:41.494+0000 7fab2a1f7640 -1 osd.2 0 waiting for initial osdmap 2026-02-19T20:07:41.496 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:07:41.494+0000 7f5c055f7640 -1 osd.0 0 waiting for initial osdmap 2026-02-19T20:07:41.497 INFO:tasks.ceph.cluster2.osd.1.trial080.stderr:2026-02-19T20:07:41.496+0000 7f1a98de7640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-19T20:07:41.498 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:07:41.496+0000 7fab2212a640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-19T20:07:41.498 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:07:41.496+0000 7f5bfd5e7640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-19T20:07:46.226 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2026-02-19T20:07:46.394 INFO:teuthology.misc.health.trial080.stdout: 2026-02-19T20:07:46.395 INFO:teuthology.misc.health.trial080.stdout:{"epoch":11,"fsid":"28c96ac4-2d59-44da-9699-cda71e572ddf","created":"2026-02-19T20:07:38.484071+0000","modified":"2026-02-19T20:07:45.503876+0000","last_up_change":"2026-02-19T20:07:42.491173+0000","last_in_change":"2026-02-19T20:07:39.160813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-19T20:07:44.383359+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e174f3ab-c860-44cb-b827-38110018fec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6800","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6801","nonce":515137397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6802","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6803","nonce":515137397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6806","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6807","nonce":515137397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6804","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6805","nonce":515137397}]},"public_addr":"10.20.193.80:6801/515137397","cluster_addr":"10.20.193.80:6803/515137397","heartbeat_back_addr":"10.20.193.80:6807/515137397","heartbeat_front_addr":"10.20.193.80:6805/515137397","state":["exists","up"]},{"osd":1,"uuid":"6d926fb6-1591-422d-8ca1-58fb499c2772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6808","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6810","nonce":2487769537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6812","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6814","nonce":2487769537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6820","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6822","nonce":2487769537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6816","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6818","nonce":2487769537}]},"public_addr":"10.20.193.80:6810/2487769537","cluster_addr":"10.20.193.80:6814/2487769537","heartbeat_back_addr":"10.20.193.80:6822/2487769537","heartbeat_front_addr":"10.20.193.80:6818/2487769537","state":["exists","up"]},{"osd":2,"uuid":"b259ad1f-166b-47e6-81f9-3e585f1bf53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6809","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6811","nonce":4168087344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6813","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6815","nonce":4168087344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6821","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6823","nonce":4168087344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6817","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6819","nonce":4168087344}]},"public_addr":"10.20.193.80:6811/4168087344","cluster_addr":"10.20.193.80:6815/4168087344","heartbeat_back_addr":"10.20.193.80:6823/4168087344","heartbeat_front_addr":"10.20.193.80:6819/4168087344","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.960240+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.940525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.959454+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:46.402 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-19T20:07:46.402 INFO:tasks.ceph:Creating RBD pool 2026-02-19T20:07:46.402 DEBUG:teuthology.orchestra.run.trial080:> sudo ceph --cluster cluster2 osd pool create rbd 8 2026-02-19T20:07:47.509 INFO:teuthology.orchestra.run.trial080.stderr:pool 'rbd' created 2026-02-19T20:07:47.522 DEBUG:teuthology.orchestra.run.trial080:> rbd --cluster cluster2 pool init rbd 2026-02-19T20:07:50.520 INFO:tasks.ceph:Starting mds daemons in cluster cluster2... 2026-02-19T20:07:50.521 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 config log 1 --format=json 2026-02-19T20:07:50.521 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-19T20:07:50.696 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:50.704 INFO:teuthology.orchestra.run.trial080.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-19T20:07:50.704 INFO:tasks.ceph_manager:config epoch is 1 2026-02-19T20:07:50.705 INFO:tasks.ceph:Waiting until cluster2 daemons up and pgs clean... 2026-02-19T20:07:50.705 INFO:tasks.ceph.ceph_manager.cluster2:waiting for mgr available 2026-02-19T20:07:50.705 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 mgr dump --format=json 2026-02-19T20:07:50.912 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:50.920 INFO:teuthology.orchestra.run.trial080.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6826","nonce":3036697833},{"type":"v1","addr":"10.20.193.73:6827","nonce":3036697833}]},"active_addr":"10.20.193.73:6827/3036697833","active_change":"2026-02-19T20:07:40.364843+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.73:0","nonce":3703206605}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.73:0","nonce":34034110}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.73:0","nonce":2875980482}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.73:0","nonce":3653889286}]}]} 2026-02-19T20:07:50.923 INFO:tasks.ceph.ceph_manager.cluster2:mgr available! 2026-02-19T20:07:50.923 INFO:tasks.ceph.ceph_manager.cluster2:waiting for all up 2026-02-19T20:07:50.923 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2026-02-19T20:07:51.101 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:51.102 INFO:teuthology.orchestra.run.trial080.stdout:{"epoch":16,"fsid":"28c96ac4-2d59-44da-9699-cda71e572ddf","created":"2026-02-19T20:07:38.484071+0000","modified":"2026-02-19T20:07:50.513120+0000","last_up_change":"2026-02-19T20:07:42.491173+0000","last_in_change":"2026-02-19T20:07:39.160813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-19T20:07:44.383359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-19T20:07:46.578473+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e174f3ab-c860-44cb-b827-38110018fec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6800","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6801","nonce":515137397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6802","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6803","nonce":515137397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6806","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6807","nonce":515137397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6804","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6805","nonce":515137397}]},"public_addr":"10.20.193.80:6801/515137397","cluster_addr":"10.20.193.80:6803/515137397","heartbeat_back_addr":"10.20.193.80:6807/515137397","heartbeat_front_addr":"10.20.193.80:6805/515137397","state":["exists","up"]},{"osd":1,"uuid":"6d926fb6-1591-422d-8ca1-58fb499c2772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6808","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6810","nonce":2487769537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6812","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6814","nonce":2487769537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6820","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6822","nonce":2487769537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6816","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6818","nonce":2487769537}]},"public_addr":"10.20.193.80:6810/2487769537","cluster_addr":"10.20.193.80:6814/2487769537","heartbeat_back_addr":"10.20.193.80:6822/2487769537","heartbeat_front_addr":"10.20.193.80:6818/2487769537","state":["exists","up"]},{"osd":2,"uuid":"b259ad1f-166b-47e6-81f9-3e585f1bf53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6809","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6811","nonce":4168087344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6813","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6815","nonce":4168087344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6821","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6823","nonce":4168087344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6817","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6819","nonce":4168087344}]},"public_addr":"10.20.193.80:6811/4168087344","cluster_addr":"10.20.193.80:6815/4168087344","heartbeat_back_addr":"10.20.193.80:6823/4168087344","heartbeat_front_addr":"10.20.193.80:6819/4168087344","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.960240+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.940525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.959454+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:51.111 INFO:tasks.ceph.ceph_manager.cluster2:all up! 2026-02-19T20:07:51.112 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2026-02-19T20:07:51.288 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:51.289 INFO:teuthology.orchestra.run.trial080.stdout:{"epoch":16,"fsid":"28c96ac4-2d59-44da-9699-cda71e572ddf","created":"2026-02-19T20:07:38.484071+0000","modified":"2026-02-19T20:07:50.513120+0000","last_up_change":"2026-02-19T20:07:42.491173+0000","last_in_change":"2026-02-19T20:07:39.160813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-19T20:07:44.383359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-19T20:07:46.578473+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e174f3ab-c860-44cb-b827-38110018fec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6800","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6801","nonce":515137397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6802","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6803","nonce":515137397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6806","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6807","nonce":515137397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6804","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6805","nonce":515137397}]},"public_addr":"10.20.193.80:6801/515137397","cluster_addr":"10.20.193.80:6803/515137397","heartbeat_back_addr":"10.20.193.80:6807/515137397","heartbeat_front_addr":"10.20.193.80:6805/515137397","state":["exists","up"]},{"osd":1,"uuid":"6d926fb6-1591-422d-8ca1-58fb499c2772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6808","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6810","nonce":2487769537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6812","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6814","nonce":2487769537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6820","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6822","nonce":2487769537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6816","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6818","nonce":2487769537}]},"public_addr":"10.20.193.80:6810/2487769537","cluster_addr":"10.20.193.80:6814/2487769537","heartbeat_back_addr":"10.20.193.80:6822/2487769537","heartbeat_front_addr":"10.20.193.80:6818/2487769537","state":["exists","up"]},{"osd":2,"uuid":"b259ad1f-166b-47e6-81f9-3e585f1bf53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6809","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6811","nonce":4168087344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6813","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6815","nonce":4168087344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6821","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6823","nonce":4168087344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6817","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6819","nonce":4168087344}]},"public_addr":"10.20.193.80:6811/4168087344","cluster_addr":"10.20.193.80:6815/4168087344","heartbeat_back_addr":"10.20.193.80:6823/4168087344","heartbeat_front_addr":"10.20.193.80:6819/4168087344","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.960240+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.940525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.959454+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:07:51.299 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 flush_pg_stats 2026-02-19T20:07:51.299 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 flush_pg_stats 2026-02-19T20:07:51.300 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 flush_pg_stats 2026-02-19T20:07:51.387 INFO:teuthology.orchestra.run.trial080.stdout:34359738371 2026-02-19T20:07:51.387 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2026-02-19T20:07:51.389 INFO:teuthology.orchestra.run.trial080.stdout:34359738371 2026-02-19T20:07:51.389 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2026-02-19T20:07:51.390 INFO:teuthology.orchestra.run.trial080.stdout:34359738371 2026-02-19T20:07:51.391 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2026-02-19T20:07:51.609 INFO:teuthology.orchestra.run.trial080.stdout:34359738370 2026-02-19T20:07:51.610 INFO:teuthology.orchestra.run.trial080.stdout:34359738370 2026-02-19T20:07:51.612 INFO:teuthology.orchestra.run.trial080.stdout:34359738370 2026-02-19T20:07:51.620 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738370 for osd.0 2026-02-19T20:07:51.621 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738370 for osd.2 2026-02-19T20:07:51.624 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738370 for osd.1 2026-02-19T20:07:52.621 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2026-02-19T20:07:52.621 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2026-02-19T20:07:52.624 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2026-02-19T20:07:52.800 INFO:teuthology.orchestra.run.trial080.stdout:34359738371 2026-02-19T20:07:52.803 INFO:teuthology.orchestra.run.trial080.stdout:34359738371 2026-02-19T20:07:52.810 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738371 for osd.2 2026-02-19T20:07:52.810 DEBUG:teuthology.parallel:result is None 2026-02-19T20:07:52.815 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738371 for osd.0 2026-02-19T20:07:52.815 DEBUG:teuthology.parallel:result is None 2026-02-19T20:07:52.842 INFO:teuthology.orchestra.run.trial080.stdout:34359738371 2026-02-19T20:07:52.852 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738371 for osd.1 2026-02-19T20:07:52.852 DEBUG:teuthology.parallel:result is None 2026-02-19T20:07:52.852 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2026-02-19T20:07:52.852 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2026-02-19T20:07:53.068 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:53.068 INFO:teuthology.orchestra.run.trial080.stderr:dumped all 2026-02-19T20:07:53.077 INFO:teuthology.orchestra.run.trial080.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2026-02-19T20:07:52.368803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81408,"kb_used_data":528,"kb_used_omap":25,"kb_used_meta":80806,"kb_avail":283034112,"statfs":{"total":289910292480,"available":289826930688,"internally_reserved":0,"allocated":540672,"data_stored":1303003,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":26325,"internal_metadata":82745643},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.860645"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516446+0000","last_change":"2026-02-19T20:07:50.516446+0000","last_active":"2026-02-19T20:07:50.516446+0000","last_peered":"2026-02-19T20:07:50.516446+0000","last_clean":"2026-02-19T20:07:50.516446+0000","last_became_active":"2026-02-19T20:07:48.515436+0000","last_became_peered":"2026-02-19T20:07:48.515436+0000","last_unstale":"2026-02-19T20:07:50.516446+0000","last_undegraded":"2026-02-19T20:07:50.516446+0000","last_fullsized":"2026-02-19T20:07:50.516446+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:20:02.908479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073110000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516622+0000","last_change":"2026-02-19T20:07:50.516622+0000","last_active":"2026-02-19T20:07:50.516622+0000","last_peered":"2026-02-19T20:07:50.516622+0000","last_clean":"2026-02-19T20:07:50.516622+0000","last_became_active":"2026-02-19T20:07:48.515323+0000","last_became_peered":"2026-02-19T20:07:48.515323+0000","last_unstale":"2026-02-19T20:07:50.516622+0000","last_undegraded":"2026-02-19T20:07:50.516622+0000","last_fullsized":"2026-02-19T20:07:50.516622+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T07:26:46.646425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091905299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516403+0000","last_change":"2026-02-19T20:07:50.516403+0000","last_active":"2026-02-19T20:07:50.516403+0000","last_peered":"2026-02-19T20:07:50.516403+0000","last_clean":"2026-02-19T20:07:50.516403+0000","last_became_active":"2026-02-19T20:07:48.515234+0000","last_became_peered":"2026-02-19T20:07:48.515234+0000","last_unstale":"2026-02-19T20:07:50.516403+0000","last_undegraded":"2026-02-19T20:07:50.516403+0000","last_fullsized":"2026-02-19T20:07:50.516403+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:29:42.841704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073914599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516159+0000","last_change":"2026-02-19T20:07:50.516159+0000","last_active":"2026-02-19T20:07:50.516159+0000","last_peered":"2026-02-19T20:07:50.516159+0000","last_clean":"2026-02-19T20:07:50.516159+0000","last_became_active":"2026-02-19T20:07:48.515367+0000","last_became_peered":"2026-02-19T20:07:48.515367+0000","last_unstale":"2026-02-19T20:07:50.516159+0000","last_undegraded":"2026-02-19T20:07:50.516159+0000","last_fullsized":"2026-02-19T20:07:50.516159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:50:02.278192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050451599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.517147+0000","last_change":"2026-02-19T20:07:50.516971+0000","last_active":"2026-02-19T20:07:50.517147+0000","last_peered":"2026-02-19T20:07:50.517147+0000","last_clean":"2026-02-19T20:07:50.517147+0000","last_became_active":"2026-02-19T20:07:48.515048+0000","last_became_peered":"2026-02-19T20:07:48.515048+0000","last_unstale":"2026-02-19T20:07:50.517147+0000","last_undegraded":"2026-02-19T20:07:50.517147+0000","last_fullsized":"2026-02-19T20:07:50.517147+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:08:18.808510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590999999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.559960+0000","last_change":"2026-02-19T20:07:50.559960+0000","last_active":"2026-02-19T20:07:50.559960+0000","last_peered":"2026-02-19T20:07:50.559960+0000","last_clean":"2026-02-19T20:07:50.559960+0000","last_became_active":"2026-02-19T20:07:48.514961+0000","last_became_peered":"2026-02-19T20:07:48.514961+0000","last_unstale":"2026-02-19T20:07:50.559960+0000","last_undegraded":"2026-02-19T20:07:50.559960+0000","last_fullsized":"2026-02-19T20:07:50.559960+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:28:50.223157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000202339,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.559971+0000","last_change":"2026-02-19T20:07:50.559971+0000","last_active":"2026-02-19T20:07:50.559971+0000","last_peered":"2026-02-19T20:07:50.559971+0000","last_clean":"2026-02-19T20:07:50.559971+0000","last_became_active":"2026-02-19T20:07:48.515007+0000","last_became_peered":"2026-02-19T20:07:48.515007+0000","last_unstale":"2026-02-19T20:07:50.559971+0000","last_undegraded":"2026-02-19T20:07:50.559971+0000","last_fullsized":"2026-02-19T20:07:50.559971+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:35:54.659935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000211387,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":21,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516119+0000","last_change":"2026-02-19T20:07:50.516119+0000","last_active":"2026-02-19T20:07:50.516119+0000","last_peered":"2026-02-19T20:07:50.516119+0000","last_clean":"2026-02-19T20:07:50.516119+0000","last_became_active":"2026-02-19T20:07:48.514767+0000","last_became_peered":"2026-02-19T20:07:48.514767+0000","last_unstale":"2026-02-19T20:07:50.516119+0000","last_undegraded":"2026-02-19T20:07:50.516119+0000","last_fullsized":"2026-02-19T20:07:50.516119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:16:18.619230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048350599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'72","reported_seq":111,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.515386+0000","last_change":"2026-02-19T20:07:45.946467+0000","last_active":"2026-02-19T20:07:50.515386+0000","last_peered":"2026-02-19T20:07:50.515386+0000","last_clean":"2026-02-19T20:07:50.515386+0000","last_became_active":"2026-02-19T20:07:45.945977+0000","last_became_peered":"2026-02-19T20:07:45.945977+0000","last_unstale":"2026-02-19T20:07:50.515386+0000","last_undegraded":"2026-02-19T20:07:50.515386+0000","last_fullsized":"2026-02-19T20:07:50.515386+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_clean_scrub_stamp":"2026-02-19T20:07:44.502565+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:33.783466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":40743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":196,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":200704,"data_stored":631130,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":9425,"internal_metadata":27581231},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":196,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":200704,"data_stored":631130,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":9425,"internal_metadata":27581231},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:07:53.078 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2026-02-19T20:07:53.251 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:53.251 INFO:teuthology.orchestra.run.trial080.stderr:dumped all 2026-02-19T20:07:53.258 INFO:teuthology.orchestra.run.trial080.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2026-02-19T20:07:52.368803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81408,"kb_used_data":528,"kb_used_omap":25,"kb_used_meta":80806,"kb_avail":283034112,"statfs":{"total":289910292480,"available":289826930688,"internally_reserved":0,"allocated":540672,"data_stored":1303003,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":26325,"internal_metadata":82745643},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.860645"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516446+0000","last_change":"2026-02-19T20:07:50.516446+0000","last_active":"2026-02-19T20:07:50.516446+0000","last_peered":"2026-02-19T20:07:50.516446+0000","last_clean":"2026-02-19T20:07:50.516446+0000","last_became_active":"2026-02-19T20:07:48.515436+0000","last_became_peered":"2026-02-19T20:07:48.515436+0000","last_unstale":"2026-02-19T20:07:50.516446+0000","last_undegraded":"2026-02-19T20:07:50.516446+0000","last_fullsized":"2026-02-19T20:07:50.516446+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:20:02.908479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073110000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516622+0000","last_change":"2026-02-19T20:07:50.516622+0000","last_active":"2026-02-19T20:07:50.516622+0000","last_peered":"2026-02-19T20:07:50.516622+0000","last_clean":"2026-02-19T20:07:50.516622+0000","last_became_active":"2026-02-19T20:07:48.515323+0000","last_became_peered":"2026-02-19T20:07:48.515323+0000","last_unstale":"2026-02-19T20:07:50.516622+0000","last_undegraded":"2026-02-19T20:07:50.516622+0000","last_fullsized":"2026-02-19T20:07:50.516622+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T07:26:46.646425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091905299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516403+0000","last_change":"2026-02-19T20:07:50.516403+0000","last_active":"2026-02-19T20:07:50.516403+0000","last_peered":"2026-02-19T20:07:50.516403+0000","last_clean":"2026-02-19T20:07:50.516403+0000","last_became_active":"2026-02-19T20:07:48.515234+0000","last_became_peered":"2026-02-19T20:07:48.515234+0000","last_unstale":"2026-02-19T20:07:50.516403+0000","last_undegraded":"2026-02-19T20:07:50.516403+0000","last_fullsized":"2026-02-19T20:07:50.516403+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:29:42.841704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073914599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516159+0000","last_change":"2026-02-19T20:07:50.516159+0000","last_active":"2026-02-19T20:07:50.516159+0000","last_peered":"2026-02-19T20:07:50.516159+0000","last_clean":"2026-02-19T20:07:50.516159+0000","last_became_active":"2026-02-19T20:07:48.515367+0000","last_became_peered":"2026-02-19T20:07:48.515367+0000","last_unstale":"2026-02-19T20:07:50.516159+0000","last_undegraded":"2026-02-19T20:07:50.516159+0000","last_fullsized":"2026-02-19T20:07:50.516159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:50:02.278192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050451599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"16'2","reported_seq":22,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.517147+0000","last_change":"2026-02-19T20:07:50.516971+0000","last_active":"2026-02-19T20:07:50.517147+0000","last_peered":"2026-02-19T20:07:50.517147+0000","last_clean":"2026-02-19T20:07:50.517147+0000","last_became_active":"2026-02-19T20:07:48.515048+0000","last_became_peered":"2026-02-19T20:07:48.515048+0000","last_unstale":"2026-02-19T20:07:50.517147+0000","last_undegraded":"2026-02-19T20:07:50.517147+0000","last_fullsized":"2026-02-19T20:07:50.517147+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:08:18.808510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590999999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.559960+0000","last_change":"2026-02-19T20:07:50.559960+0000","last_active":"2026-02-19T20:07:50.559960+0000","last_peered":"2026-02-19T20:07:50.559960+0000","last_clean":"2026-02-19T20:07:50.559960+0000","last_became_active":"2026-02-19T20:07:48.514961+0000","last_became_peered":"2026-02-19T20:07:48.514961+0000","last_unstale":"2026-02-19T20:07:50.559960+0000","last_undegraded":"2026-02-19T20:07:50.559960+0000","last_fullsized":"2026-02-19T20:07:50.559960+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:28:50.223157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000202339,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.559971+0000","last_change":"2026-02-19T20:07:50.559971+0000","last_active":"2026-02-19T20:07:50.559971+0000","last_peered":"2026-02-19T20:07:50.559971+0000","last_clean":"2026-02-19T20:07:50.559971+0000","last_became_active":"2026-02-19T20:07:48.515007+0000","last_became_peered":"2026-02-19T20:07:48.515007+0000","last_unstale":"2026-02-19T20:07:50.559971+0000","last_undegraded":"2026-02-19T20:07:50.559971+0000","last_fullsized":"2026-02-19T20:07:50.559971+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:35:54.659935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000211387,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"14'1","reported_seq":21,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.516119+0000","last_change":"2026-02-19T20:07:50.516119+0000","last_active":"2026-02-19T20:07:50.516119+0000","last_peered":"2026-02-19T20:07:50.516119+0000","last_clean":"2026-02-19T20:07:50.516119+0000","last_became_active":"2026-02-19T20:07:48.514767+0000","last_became_peered":"2026-02-19T20:07:48.514767+0000","last_unstale":"2026-02-19T20:07:50.516119+0000","last_undegraded":"2026-02-19T20:07:50.516119+0000","last_fullsized":"2026-02-19T20:07:50.516119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:16:18.619230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048350599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"11'72","reported_seq":111,"reported_epoch":16,"state":"active+clean","last_fresh":"2026-02-19T20:07:50.515386+0000","last_change":"2026-02-19T20:07:45.946467+0000","last_active":"2026-02-19T20:07:50.515386+0000","last_peered":"2026-02-19T20:07:50.515386+0000","last_clean":"2026-02-19T20:07:50.515386+0000","last_became_active":"2026-02-19T20:07:45.945977+0000","last_became_peered":"2026-02-19T20:07:45.945977+0000","last_unstale":"2026-02-19T20:07:50.515386+0000","last_undegraded":"2026-02-19T20:07:50.515386+0000","last_fullsized":"2026-02-19T20:07:50.515386+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_clean_scrub_stamp":"2026-02-19T20:07:44.502565+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:33.783466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27096,"kb_used_data":136,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":94344744,"statfs":{"total":96636764160,"available":96609017856,"internally_reserved":0,"allocated":139264,"data_stored":40743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":196,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":200704,"data_stored":631130,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":9425,"internal_metadata":27581231},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":196,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":200704,"data_stored":631130,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":9425,"internal_metadata":27581231},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:07:53.260 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2026-02-19T20:07:53.260 INFO:tasks.ceph:Waiting until ceph cluster cluster2 is healthy... 2026-02-19T20:07:53.260 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy 2026-02-19T20:07:53.260 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 health --format=json 2026-02-19T20:07:53.463 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:07:53.464 INFO:teuthology.orchestra.run.trial080.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-19T20:07:53.473 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy done 2026-02-19T20:07:53.473 INFO:teuthology.run_tasks:Running task exec... 2026-02-19T20:07:53.485 INFO:teuthology.task.exec:Executing custom commands... 2026-02-19T20:07:53.485 INFO:teuthology.task.exec:Running commands on role cluster1.client.mirror.0 host ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:07:53.485 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:53.707 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror] 2026-02-19T20:07:53.707 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBtbZdpUY1gGxAAAXlSnZ0YVolJQcugW7fcPg== 2026-02-19T20:07:53.707 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror-peer" 2026-02-19T20:07:53.707 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:53.707 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror 2026-02-19T20:07:53.716 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:53.942 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.0] 2026-02-19T20:07:53.942 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBtbZdp4HhcIBAAUgpYwrAOLI9teLNvsPIukQ== 2026-02-19T20:07:53.942 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror" 2026-02-19T20:07:53.942 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:53.943 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror.0 2026-02-19T20:07:53.951 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:54.177 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.1] 2026-02-19T20:07:54.177 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBtbZdpa59aJRAA0jbVFxzazmDaWlHMpXI0Vw== 2026-02-19T20:07:54.177 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror" 2026-02-19T20:07:54.177 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:54.177 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror.1 2026-02-19T20:07:54.186 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:54.411 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.2] 2026-02-19T20:07:54.411 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBtbZdppRFzKhAA0f7hYs4eiKKPRJ+A0LxLGQ== 2026-02-19T20:07:54.411 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror" 2026-02-19T20:07:54.412 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:54.412 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror.2 2026-02-19T20:07:54.421 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:54.662 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.3] 2026-02-19T20:07:54.663 INFO:teuthology.orchestra.run.trial080.stdout: key = AQBtbZdpvuhjLxAAuP/6RH3v3TGDHFAUKPgD/Q== 2026-02-19T20:07:54.663 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror" 2026-02-19T20:07:54.663 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:54.663 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror.3 2026-02-19T20:07:54.673 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:54.900 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror] 2026-02-19T20:07:54.900 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpfOHlHBAAM9UFnLGWD3/uZXWtULyhfg== 2026-02-19T20:07:54.900 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror-peer" 2026-02-19T20:07:54.900 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:54.901 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror 2026-02-19T20:07:54.910 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:55.135 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.0] 2026-02-19T20:07:55.136 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpmXvuIRAAcqpb7qkvG/nxupuzkZEDtQ== 2026-02-19T20:07:55.136 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror" 2026-02-19T20:07:55.136 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:55.136 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror.0 2026-02-19T20:07:55.146 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:55.369 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.1] 2026-02-19T20:07:55.369 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpJ4jsJhAArKraHfk1U2Gux6CW5H2Qrg== 2026-02-19T20:07:55.369 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror" 2026-02-19T20:07:55.369 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:55.370 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror.1 2026-02-19T20:07:55.380 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:55.605 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.2] 2026-02-19T20:07:55.606 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdp85/TKxAAdDfJThAAvEFVc1zwEwe5gA== 2026-02-19T20:07:55.606 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror" 2026-02-19T20:07:55.606 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:55.606 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror.2 2026-02-19T20:07:55.615 DEBUG:teuthology.orchestra.run.trial080:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2026-02-19T20:07:55.841 INFO:teuthology.orchestra.run.trial080.stdout:[client.mirror.3] 2026-02-19T20:07:55.841 INFO:teuthology.orchestra.run.trial080.stdout: key = AQCDbZdpgKLmMBAA0x81c0AFYbOH9eIMOV+ftA== 2026-02-19T20:07:55.842 INFO:teuthology.orchestra.run.trial080.stdout: caps mon = "profile rbd-mirror" 2026-02-19T20:07:55.842 INFO:teuthology.orchestra.run.trial080.stdout: caps osd = "profile rbd" 2026-02-19T20:07:55.842 INFO:teuthology.orchestra.run.trial080.stderr:updated caps for client.mirror.3 2026-02-19T20:07:55.851 INFO:teuthology.run_tasks:Running task rbd-mirror... 2026-02-19T20:07:55.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2026-02-19T20:07:55.871 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2026-02-19T20:07:55.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2026-02-19T20:07:55.873 INFO:teuthology.run_tasks:Running task rbd-mirror... 2026-02-19T20:07:55.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2026-02-19T20:07:55.889 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2026-02-19T20:07:55.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2026-02-19T20:07:55.891 INFO:teuthology.run_tasks:Running task rbd-mirror... 2026-02-19T20:07:55.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2026-02-19T20:07:55.906 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2026-02-19T20:07:55.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2026-02-19T20:07:55.909 INFO:teuthology.run_tasks:Running task rbd-mirror... 2026-02-19T20:07:55.924 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2026-02-19T20:07:55.924 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2026-02-19T20:07:55.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2026-02-19T20:07:55.927 INFO:teuthology.run_tasks:Running task rbd-mirror... 2026-02-19T20:07:55.942 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2026-02-19T20:07:55.943 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2026-02-19T20:07:55.945 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2026-02-19T20:07:55.945 INFO:teuthology.run_tasks:Running task rbd-mirror... 2026-02-19T20:07:56.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2026-02-19T20:07:56.008 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2026-02-19T20:07:56.010 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2026-02-19T20:07:56.010 INFO:teuthology.run_tasks:Running task rbd-mirror... 2026-02-19T20:07:56.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2026-02-19T20:07:56.022 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2026-02-19T20:07:56.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2026-02-19T20:07:56.024 INFO:teuthology.run_tasks:Running task rbd-mirror... 2026-02-19T20:07:56.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2026-02-19T20:07:56.037 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2026-02-19T20:07:56.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2026-02-19T20:07:56.038 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2026-02-19T20:07:56.053 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1771531676 2026-02-19T20:07:56.053 DEBUG:tasks.rbd_mirror_thrash:Yielding 2026-02-19T20:07:56.053 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2026-02-19T20:07:56.065 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1771531676 2026-02-19T20:07:56.066 DEBUG:tasks.rbd_mirror_thrash:Yielding 2026-02-19T20:07:56.066 INFO:teuthology.run_tasks:Running task workunit... 2026-02-19T20:07:56.085 INFO:tasks.workunit:Pulling workunits from ref 802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T20:07:56.085 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-19T20:07:56.085 DEBUG:teuthology.orchestra.run.trial080:> stat -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2026-02-19T20:07:56.086 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster1 2026-02-19T20:07:56.086 INFO:tasks.rbd_mirror_thrash:waiting for 89.9695260019412 secs before thrashing 2026-02-19T20:07:56.086 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster2 2026-02-19T20:07:56.086 INFO:tasks.rbd_mirror_thrash:waiting for 66.29515119006032 secs before thrashing 2026-02-19T20:07:56.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T20:07:56.102 INFO:teuthology.orchestra.run.trial080.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.cluster1.mirror': No such file or directory 2026-02-19T20:07:56.102 DEBUG:teuthology.orchestra.run.trial080:> mkdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2026-02-19T20:07:56.159 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.cluster1.mirror 2026-02-19T20:07:56.159 DEBUG:teuthology.orchestra.run.trial080:> cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror && mkdir -- client.mirror 2026-02-19T20:07:56.215 INFO:tasks.workunit:timeout=6h 2026-02-19T20:07:56.215 INFO:tasks.workunit:cleanup=True 2026-02-19T20:07:56.215 DEBUG:teuthology.orchestra.run.trial080:> rm -rf /home/ubuntu/cephtest/clone.cluster1.client.mirror && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.cluster1.client.mirror && cd /home/ubuntu/cephtest/clone.cluster1.client.mirror && git checkout 802bf111cd55e57f5842153d7ba2b8b964aa818a 2026-02-19T20:07:56.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Cloning into '/home/ubuntu/cephtest/clone.cluster1.client.mirror'... 2026-02-19T20:08:16.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Note: switching to '802bf111cd55e57f5842153d7ba2b8b964aa818a'. 2026-02-19T20:08:16.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:08:16.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-19T20:08:16.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:state without impacting any branches by switching back to a branch. 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: git switch -c 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Or undo this operation with: 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:08:16.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: git switch - 2026-02-19T20:08:16.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:08:16.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-19T20:08:16.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:08:16.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:HEAD is now at 802bf111cd5 Merge branch 'wip-common-ceph_json-jfw-spirit-to-boost' of https://github.com/chardan/ceph into wip-yuri-testing-2026-02-18-1820 2026-02-19T20:08:16.286 DEBUG:teuthology.orchestra.run.trial080:> cd -- /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror 2026-02-19T20:08:16.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-19T20:08:16.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2026-02-19T20:08:16.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-19T20:08:16.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-19T20:08:16.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-19T20:08:16.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2026-02-19T20:08:16.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2026-02-19T20:08:16.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-19T20:08:16.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2026-02-19T20:08:16.530 DEBUG:teuthology.orchestra.run.trial080:> set -ex 2026-02-19T20:08:16.530 DEBUG:teuthology.orchestra.run.trial080:> dd if=/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror of=/dev/stdout 2026-02-19T20:08:16.587 INFO:tasks.workunit:Running workunits matching rbd/rbd_mirror_stress.sh on cluster1.client.mirror... 2026-02-19T20:08:16.590 INFO:tasks.workunit:Running workunit rbd/rbd_mirror_stress.sh... 2026-02-19T20:08:16.590 DEBUG:teuthology.orchestra.run.trial080:workunit test rbd/rbd_mirror_stress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=802bf111cd55e57f5842153d7ba2b8b964aa818a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_MNT=/home/ubuntu/cephtest/mnt.cluster1.mirror CEPH_ARGS='' MIRROR_POOL_MODE=image RBD_IMAGE_FEATURES=layering RBD_MIRROR_INSTANCES=4 RBD_MIRROR_MODE=snapshot RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2026-02-19T20:08:16.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ IMAGE_COUNT=50 2026-02-19T20:08:16.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ export LOCKDEP=0 2026-02-19T20:08:16.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ LOCKDEP=0 2026-02-19T20:08:16.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2026-02-19T20:08:16.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ . /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_helpers.sh 2026-02-19T20:08:16.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ RBD_MIRROR_INSTANCES=4 2026-02-19T20:08:16.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CLUSTER1=cluster1 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CLUSTER2=cluster2 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ PEER_CLUSTER_SUFFIX= 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ POOL=mirror 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ PARENT_POOL=mirror_parent 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ NS1=ns1 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ NS2=ns2 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ TEMPDIR= 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ID=mirror 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ RBD_IMAGE_FEATURES=layering 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ MIRROR_USER_ID_PREFIX=mirror. 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ RBD_MIRROR_MODE=snapshot 2026-02-19T20:08:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ MIRROR_POOL_MODE=image 2026-02-19T20:08:16.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' snapshot = snapshot ']' 2026-02-19T20:08:16.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ MIRROR_POOL_MODE=image 2026-02-19T20:08:16.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ export 'CEPH_ARGS=--id mirror' 2026-02-19T20:08:16.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS='--id mirror' 2026-02-19T20:08:16.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ LAST_MIRROR_INSTANCE=3 2026-02-19T20:08:16.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2026-02-19T20:08:16.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ readlink -f /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/../../../src 2026-02-19T20:08:16.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror/src 2026-02-19T20:08:16.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_BIN=. 2026-02-19T20:08:16.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_SRC=. 2026-02-19T20:08:16.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' -e CMakeCache.txt ']' 2026-02-19T20:08:16.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 -gt 0 ']' 2026-02-19T20:08:16.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ setup 2026-02-19T20:08:16.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local c 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ trap 'cleanup $?' INT TERM EXIT 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ setup_tempdir 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mktemp -d 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ TEMPDIR=/tmp/tmp.9BL5XQhnTi 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -z 1 ']' 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ setup_pools cluster1 cluster2 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_cluster=cluster2 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mon_map_file 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mon_addr 2026-02-19T20:08:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local admin_key_file 2026-02-19T20:08:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local uuid 2026-02-19T20:08:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ ceph --cluster cluster1 osd pool create mirror 64 64 2026-02-19T20:08:17.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:pool 'mirror' already exists 2026-02-19T20:08:17.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:17.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ ceph --cluster cluster1 osd pool create mirror_parent 64 64 2026-02-19T20:08:18.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:pool 'mirror_parent' already exists 2026-02-19T20:08:18.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:18.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 pool init mirror 2026-02-19T20:08:21.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:21.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 pool init mirror_parent 2026-02-19T20:08:24.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:24.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:24.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 mirror pool enable --site-name cluster1 mirror image 2026-02-19T20:08:24.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent image 2026-02-19T20:08:24.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 namespace create mirror/ns1 2026-02-19T20:08:24.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 namespace create mirror/ns2 2026-02-19T20:08:24.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 namespace create mirror_parent/ns1 2026-02-19T20:08:24.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns1 image 2026-02-19T20:08:24.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns2 image 2026-02-19T20:08:24.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent/ns1 image 2026-02-19T20:08:24.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -z ']' 2026-02-19T20:08:24.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -z ']' 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ peer_add cluster1 mirror cluster2 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local client_cluster=cluster2 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_cluster=cluster2 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local uuid_var_name 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:24.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local error_code 2026-02-19T20:08:24.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local peer_uuid 2026-02-19T20:08:24.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 16 32 2026-02-19T20:08:24.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:08:24.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool peer add mirror cluster2 2026-02-19T20:08:24.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ peer_uuid=8a4af866-9c74-424e-bb26-1d9409d4e8b9 2026-02-19T20:08:24.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:08:24.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 17 ']' 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 -eq 0 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ peer_add cluster1 mirror_parent cluster2 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror_parent 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local client_cluster=cluster2 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:24.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_cluster=cluster2 2026-02-19T20:08:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local uuid_var_name 2026-02-19T20:08:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local error_code 2026-02-19T20:08:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local peer_uuid 2026-02-19T20:08:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 16 32 2026-02-19T20:08:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:08:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool peer add mirror_parent cluster2 2026-02-19T20:08:24.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ peer_uuid=6cd40451-96e6-4f53-89d0-dcca28c6042d 2026-02-19T20:08:24.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:08:24.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:08:24.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 17 ']' 2026-02-19T20:08:24.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 -eq 0 2026-02-19T20:08:24.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:24.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:08:24.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ setup_pools cluster2 cluster1 2026-02-19T20:08:24.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:08:24.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_cluster=cluster1 2026-02-19T20:08:24.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mon_map_file 2026-02-19T20:08:24.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mon_addr 2026-02-19T20:08:24.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local admin_key_file 2026-02-19T20:08:24.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local uuid 2026-02-19T20:08:24.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:24.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ ceph --cluster cluster2 osd pool create mirror 64 64 2026-02-19T20:08:25.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:pool 'mirror' already exists 2026-02-19T20:08:25.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:25.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ ceph --cluster cluster2 osd pool create mirror_parent 64 64 2026-02-19T20:08:26.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:08:26.448+0000 7fe258b699c0 -1 rbd::mirror::PoolReplayer: 0x41f0000 init: failed to initialize remote pool poller: (2) No such file or directory 2026-02-19T20:08:26.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:08:26.448+0000 7f42f968c9c0 -1 rbd::mirror::PoolReplayer: 0x4122000 init: failed to initialize remote pool poller: (2) No such file or directory 2026-02-19T20:08:26.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:08:26.448+0000 7f1ecaee69c0 -1 rbd::mirror::PoolReplayer: 0x3d16000 init: failed to initialize remote pool poller: (2) No such file or directory 2026-02-19T20:08:26.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:08:26.448+0000 7f6e26bca9c0 -1 rbd::mirror::PoolReplayer: 0x3c0c000 init: failed to initialize remote pool poller: (2) No such file or directory 2026-02-19T20:08:26.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:pool 'mirror_parent' already exists 2026-02-19T20:08:26.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:26.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 pool init mirror 2026-02-19T20:08:27.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:08:27.454+0000 7f42f968c9c0 -1 rbd::mirror::PoolReplayer: 0x41da900 init: failed to initialize remote pool poller: (2) No such file or directory 2026-02-19T20:08:27.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:08:27.454+0000 7f6e26bca9c0 -1 rbd::mirror::PoolReplayer: 0x3ce4900 init: failed to initialize remote pool poller: (2) No such file or directory 2026-02-19T20:08:27.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:08:27.454+0000 7fe258b699c0 -1 rbd::mirror::PoolReplayer: 0x5022900 init: failed to initialize remote pool poller: (2) No such file or directory 2026-02-19T20:08:27.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:08:27.456+0000 7f1ecaee69c0 -1 rbd::mirror::PoolReplayer: 0x3dd2900 init: failed to initialize remote pool poller: (2) No such file or directory 2026-02-19T20:08:29.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:29.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 pool init mirror_parent 2026-02-19T20:08:32.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:32.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:08:32.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror pool enable --site-name cluster2 mirror image 2026-02-19T20:08:32.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent image 2026-02-19T20:08:32.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 namespace create mirror/ns1 2026-02-19T20:08:32.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 namespace create mirror/ns2 2026-02-19T20:08:32.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 namespace create mirror_parent/ns1 2026-02-19T20:08:32.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns1 image 2026-02-19T20:08:32.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns2 image 2026-02-19T20:08:32.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent/ns1 image 2026-02-19T20:08:32.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -z ']' 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -z ']' 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ peer_add cluster2 mirror cluster1 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local client_cluster=cluster1 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_cluster=cluster1 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local uuid_var_name 2026-02-19T20:08:32.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:32.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local error_code 2026-02-19T20:08:32.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local peer_uuid 2026-02-19T20:08:32.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 16 32 2026-02-19T20:08:32.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:08:32.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster2 mirror pool peer add mirror cluster1 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ peer_uuid=fcd85b27-4f6c-49d0-b6a1-c0f0528fce3c 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 17 ']' 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 -eq 0 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ peer_add cluster2 mirror_parent cluster1 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:08:32.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror_parent 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local client_cluster=cluster1 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_cluster=cluster1 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local uuid_var_name 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local error_code 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local peer_uuid 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 16 32 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:08:32.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster2 mirror pool peer add mirror_parent cluster1 2026-02-19T20:08:32.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ peer_uuid=fe499954-2f57-421a-a8f6-42974be5dfb1 2026-02-19T20:08:32.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:08:32.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:08:32.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 17 ']' 2026-02-19T20:08:32.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 -eq 0 2026-02-19T20:08:32.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:32.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:08:32.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:08:32.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirrors cluster1 2026-02-19T20:08:32.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:08:32.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 0 3 2026-02-19T20:08:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirror cluster1:0 2026-02-19T20:08:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1:0 2026-02-19T20:08:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance 2026-02-19T20:08:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:08:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local val=cluster1:0 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster_var_name=cluster 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance_var_name=instance 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cluster=cluster1 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=0 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval cluster=cluster1 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval instance=0 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:08:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirror cluster1:1 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1:1 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local val=cluster1:1 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster_var_name=cluster 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance_var_name=instance 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cluster=cluster1 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=1 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval cluster=cluster1 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval instance=1 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirror cluster1:2 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1:2 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local val=cluster1:2 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster_var_name=cluster 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance_var_name=instance 2026-02-19T20:08:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cluster=cluster1 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=2 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval cluster=cluster1 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval instance=2 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirror cluster1:3 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1:3 2026-02-19T20:08:32.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local val=cluster1:3 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster_var_name=cluster 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance_var_name=instance 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cluster=cluster1 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=3 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval cluster=cluster1 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval instance=3 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:08:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirrors cluster2 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 0 3 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirror cluster2:0 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2:0 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_cluster_instance cluster2:0 cluster instance 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local val=cluster2:0 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster_var_name=cluster 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance_var_name=instance 2026-02-19T20:08:32.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cluster=cluster2 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=0 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 = cluster2:0 ']' 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval cluster=cluster2 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster2 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval instance=0 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirror cluster2:1 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2:1 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance 2026-02-19T20:08:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_cluster_instance cluster2:1 cluster instance 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local val=cluster2:1 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster_var_name=cluster 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance_var_name=instance 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cluster=cluster2 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=1 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 1 = cluster2:1 ']' 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval cluster=cluster2 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster2 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval instance=1 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:08:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirror cluster2:2 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2:2 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_cluster_instance cluster2:2 cluster instance 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local val=cluster2:2 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster_var_name=cluster 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance_var_name=instance 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cluster=cluster2 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=2 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 2 = cluster2:2 ']' 2026-02-19T20:08:32.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval cluster=cluster2 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster2 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval instance=2 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirror cluster2:3 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2:3 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_cluster_instance cluster2:3 cluster instance 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local val=cluster2:3 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster_var_name=cluster 2026-02-19T20:08:32.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance_var_name=instance 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cluster=cluster2 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=3 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 3 = cluster2:3 ']' 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval cluster=cluster2 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster2 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval instance=3 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ testlog 'TEST: add image and test replay after client crashes' 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ tee -a /tmp/tmp.9BL5XQhnTi/rbd-mirror.test.log 2026-02-19T20:08:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ date '+%F %T' 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo 2026-02-19 20:08:32 TEST: add image and test replay after client crashes 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:2026-02-19 20:08:32 TEST: add image and test replay after client crashes 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=test 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror test snapshot 512M 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:08:32.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror test 512M 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:08:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/test 2026-02-19T20:08:32.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:08:32.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:32.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:08:32.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:08:32.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror test snapshot 2026-02-19T20:08:32.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:08:32.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:32.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:08:32.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:08:32.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test snapshot 2026-02-19T20:08:33.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:08:33.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/test 2026-02-19T20:08:33.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:08:33.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10c02608b715 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10c02608b715 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:32 2026 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:08:32 2026 2026-02-19T20:08:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:32 2026 2026-02-19T20:08:33.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:08:33.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:08:33.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:08:33.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:08:33.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:08:33.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:08:33.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:33.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:08:33.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:08:33.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:08:33.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:33.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:08:33.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:08:33.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:08:33.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:08:33.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:08:34.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:08:34.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:08:34.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:34.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:08:34.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:08:34.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:08:34.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:08:34.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:08:34.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:08:34.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:08:34.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:34.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:08:34.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:08:34.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:08:34.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:08:34.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:08:34.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:08:34.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:08:34.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:08:34.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:34.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:34.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:34.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:34.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:34.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:34.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:34.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:08:34.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:08:34.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:08:34.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:34.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:34.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:34.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:08:34.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:34.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:34.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:34.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:34.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:08:34.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:08:34.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:08:34.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:34.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:34.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:08:34.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:08:34.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:08:34.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:34.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:34.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:34.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:08:34.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:34.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:34.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:34.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:34.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:34.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:34.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:34.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:34.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:34.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:34.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:34.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:08:34.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:08:34.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:08:34.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:34.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:34.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:34.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:08:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:34.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:34.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:34.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2026-02-19T20:08:34.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:08:34.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:34.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:34.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:08:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:08:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:08:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:08:34.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:34.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:34.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:34.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2026-02-19T20:08:34.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:34.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:34.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:34.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:08:34.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 1 2026-02-19T20:08:35.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:08:35.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:08:35.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:35.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:08:35.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:08:35.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:08:35.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:08:35.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:35.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:35.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:35.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:08:35.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:35.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:35.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:35.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:35.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:08:35.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:08:35.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:35.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:35.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:35.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:08:35.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:08:35.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:08:35.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:35.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:35.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:35.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:08:35.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:35.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:35.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:35.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:35.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:35.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:35.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:35.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:35.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:08:35.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:08:35.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:08:35.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:35.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:35.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:35.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:08:35.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:35.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:35.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:35.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:35.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2026-02-19T20:08:35.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:35.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:35.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:35.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:35.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:08:35.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:08:35.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:08:35.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:35.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:35.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2026-02-19T20:08:35.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:35.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:35.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:35.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:08:35.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 2 2026-02-19T20:08:37.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:08:37.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:08:37.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:08:37.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:37.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:37.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:37.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:08:37.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:37.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:37.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:37.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:37.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:08:37.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:08:37.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:08:37.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:37.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:37.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:37.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:08:38.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:38.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:38.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:38.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:38.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:08:38.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:08:38.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:38.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:38.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:38.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:38.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:08:38.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:08:38.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:08:38.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:38.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:38.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:38.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:08:38.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:38.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:38.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:38.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:38.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2026-02-19T20:08:38.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:08:38.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:38.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:38.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:08:38.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:08:38.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:38.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:08:38.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:08:38.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:38.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:38.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:38.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2026-02-19T20:08:38.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:38.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:38.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:38.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:08:38.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 4 2026-02-19T20:08:42.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:08:42.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:08:42.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:08:42.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:08:42.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:08:42.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:42.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:42.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:42.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:08:42.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:42.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:42.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:42.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:08:42.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:08:42.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:08:42.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:42.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:42.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:42.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:08:42.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:42.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:42.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:42.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:08:42.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:08:42.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:08:42.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:42.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:42.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:42.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:08:42.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:42.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:42.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:08:42.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:08:42.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:08:42.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:42.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:42.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:42.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2026-02-19T20:08:42.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:42.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:42.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:42.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:08:42.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 8 2026-02-19T20:08:50.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:08:50.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:50.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:08:50.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:08:50.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:08:50.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:50.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:50.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:50.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:08:50.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:50.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:50.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:50.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:50.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:08:50.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:08:50.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:50.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:50.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:08:50.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:08:50.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:50.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:08:50.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:08:50.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:50.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:50.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:50.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:08:50.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:50.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:50.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:50.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:50.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:08:50.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:08:50.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:50.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:50.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:08:50.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:08:50.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:50.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:50.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:50.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:08:50.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:50.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:50.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:50.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:08:50.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:50.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:08:50.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:08:50.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:50.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:50.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:08:50.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:08:50.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:50.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:50.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:50.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:50.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:50.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:50.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:50.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:08:50.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:08:50.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:08:50.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:50.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:50.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:50.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:08:50.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:50.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:50.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:50.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:50.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/test 2026-02-19T20:08:50.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:08:50.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:50.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:50.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:08:50.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:08:50.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:50.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:08:50.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:08:50.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:08:50.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:50.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:50.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:08:50.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2026-02-19T20:08:50.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:50.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:50.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:50.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:08:50.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 8 2026-02-19T20:08:56.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:08:56.460+0000 7f6e26bca9c0 -1 rbd::mirror::Mirror: 0x2d04360 update_pool_replayers: restarting failed pool replayer for uuid: 8a4af866-9c74-424e-bb26-1d9409d4e8b9 cluster: cluster2 client: client.mirror 2026-02-19T20:08:56.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:08:56.460+0000 7fe258b699c0 -1 rbd::mirror::Mirror: 0x32be360 update_pool_replayers: restarting failed pool replayer for uuid: 8a4af866-9c74-424e-bb26-1d9409d4e8b9 cluster: cluster2 client: client.mirror 2026-02-19T20:08:56.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:08:56.460+0000 7f42f968c9c0 -1 rbd::mirror::Mirror: 0x31f8360 update_pool_replayers: restarting failed pool replayer for uuid: 8a4af866-9c74-424e-bb26-1d9409d4e8b9 cluster: cluster2 client: client.mirror 2026-02-19T20:08:56.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:08:56.461+0000 7f1ecaee69c0 -1 rbd::mirror::Mirror: 0x2e0e360 update_pool_replayers: restarting failed pool replayer for uuid: 8a4af866-9c74-424e-bb26-1d9409d4e8b9 cluster: cluster2 client: client.mirror 2026-02-19T20:08:56.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:08:56.482+0000 7f42f968c9c0 -1 rbd::mirror::Mirror: 0x31f8360 update_pool_replayers: restarting failed pool replayer for uuid: 6cd40451-96e6-4f53-89d0-dcca28c6042d cluster: cluster2 client: client.mirror 2026-02-19T20:08:56.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:08:56.484+0000 7f6e26bca9c0 -1 rbd::mirror::Mirror: 0x2d04360 update_pool_replayers: restarting failed pool replayer for uuid: 6cd40451-96e6-4f53-89d0-dcca28c6042d cluster: cluster2 client: client.mirror 2026-02-19T20:08:56.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:08:56.487+0000 7f1ecaee69c0 -1 rbd::mirror::Mirror: 0x2e0e360 update_pool_replayers: restarting failed pool replayer for uuid: 6cd40451-96e6-4f53-89d0-dcca28c6042d cluster: cluster2 client: client.mirror 2026-02-19T20:08:56.488 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:08:56.487+0000 7fe258b699c0 -1 rbd::mirror::Mirror: 0x32be360 update_pool_replayers: restarting failed pool replayer for uuid: 6cd40451-96e6-4f53-89d0-dcca28c6042d cluster: cluster2 client: client.mirror 2026-02-19T20:08:58.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:08:58.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:08:58.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:58.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:08:58.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:08:58.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:58.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:58.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:58.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:58.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:58.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:08:58.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:08:58.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:08:58.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:58.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:58.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:08:58.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:08:58.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:58.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:58.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:58.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:08:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:08:58.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:58.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:58.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:08:58.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:08:58.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:08:58.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:08:58.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:08:58.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:08:58.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:08:58.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:08:58.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:08:58.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:08:58.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:08:58.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:08:58.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:08:58.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:08:58.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:08:58.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:08:58.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:08:58.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:58.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:58.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:08:58.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:08:58.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:08:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:08:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:08:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:08:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:08:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:08:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:08:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:08:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name= 2026-02-19T20:08:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 10 2026-02-19T20:08:58.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:08:58.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:08:58.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:08:58.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:08:58.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:08:58.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:08:58.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=14 2026-02-19T20:08:58.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:08:58.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:08:58.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 23004160 bytes 2026-02-19T20:08:58.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 46956544 bytes 2026-02-19T20:08:58.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 84547584 bytes 2026-02-19T20:08:58.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 142818304 bytes 2026-02-19T20:08:58.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 239075328 bytes 2026-02-19T20:08:59.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 308019200 bytes 2026-02-19T20:08:59.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 341665792 bytes 2026-02-19T20:08:59.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 360071168 bytes 2026-02-19T20:08:59.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 411020288 bytes 2026-02-19T20:08:59.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 450183168 bytes 2026-02-19T20:08:59.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 519201792 bytes 2026-02-19T20:08:59.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 550822912 bytes 2026-02-19T20:08:59.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 631871488 bytes 2026-02-19T20:08:59.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 640794624 bytes 2026-02-19T20:08:59.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 719620096 bytes 2026-02-19T20:08:59.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 735197184 bytes 2026-02-19T20:08:59.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 749361152 bytes 2026-02-19T20:08:59.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 824524800 bytes 2026-02-19T20:08:59.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 856371200 bytes 2026-02-19T20:08:59.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 895550464 bytes 2026-02-19T20:08:59.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 947367936 bytes 2026-02-19T20:08:59.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 977069056 bytes 2026-02-19T20:08:59.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1004787712 bytes 2026-02-19T20:08:59.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1052659712 bytes 2026-02-19T20:08:59.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1115053056 bytes 2026-02-19T20:08:59.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1189051392 bytes 2026-02-19T20:08:59.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1229983744 bytes 2026-02-19T20:08:59.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1299643392 bytes 2026-02-19T20:08:59.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1347879936 bytes 2026-02-19T20:09:00.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1409957888 bytes 2026-02-19T20:09:00.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1463176192 bytes 2026-02-19T20:09:00.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1478595584 bytes 2026-02-19T20:09:00.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1571149824 bytes 2026-02-19T20:09:00.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1650045952 bytes 2026-02-19T20:09:00.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1736883200 bytes 2026-02-19T20:09:00.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1775863808 bytes 2026-02-19T20:09:00.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1821494272 bytes 2026-02-19T20:09:00.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1822090240 bytes 2026-02-19T20:09:00.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1883064320 bytes 2026-02-19T20:09:00.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1941355520 bytes 2026-02-19T20:09:00.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1985162240 bytes 2026-02-19T20:09:00.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1986877440 bytes 2026-02-19T20:09:00.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2052982784 bytes 2026-02-19T20:09:00.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2141684736 bytes 2026-02-19T20:09:00.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2157103104 bytes 2026-02-19T20:09:00.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2212151296 bytes 2026-02-19T20:09:00.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2212768768 bytes 2026-02-19T20:09:00.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2291313664 bytes 2026-02-19T20:09:00.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2365068288 bytes 2026-02-19T20:09:00.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2412115968 bytes 2026-02-19T20:09:01.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2431093760 bytes 2026-02-19T20:09:01.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2463029248 bytes 2026-02-19T20:09:01.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2502041600 bytes 2026-02-19T20:09:01.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2528779264 bytes 2026-02-19T20:09:01.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2572669952 bytes 2026-02-19T20:09:01.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2618774528 bytes 2026-02-19T20:09:01.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2645338112 bytes 2026-02-19T20:09:01.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2695200768 bytes 2026-02-19T20:09:01.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2729598976 bytes 2026-02-19T20:09:01.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2779131904 bytes 2026-02-19T20:09:01.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2817147904 bytes 2026-02-19T20:09:01.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2817764352 bytes 2026-02-19T20:09:01.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2838039552 bytes 2026-02-19T20:09:01.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2939087872 bytes 2026-02-19T20:09:01.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2980032512 bytes 2026-02-19T20:09:01.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3013063680 bytes 2026-02-19T20:09:01.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3061002240 bytes 2026-02-19T20:09:01.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3107492864 bytes 2026-02-19T20:09:01.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3201293312 bytes 2026-02-19T20:09:01.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3226461184 bytes 2026-02-19T20:09:01.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3290993664 bytes 2026-02-19T20:09:01.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3342671872 bytes 2026-02-19T20:09:01.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3367980032 bytes 2026-02-19T20:09:02.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3475189760 bytes 2026-02-19T20:09:02.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3555835904 bytes 2026-02-19T20:09:02.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3646710784 bytes 2026-02-19T20:09:02.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3707173888 bytes 2026-02-19T20:09:02.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3743915008 bytes 2026-02-19T20:09:02.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3753390080 bytes 2026-02-19T20:09:02.292 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3775953920 bytes 2026-02-19T20:09:02.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3836045312 bytes 2026-02-19T20:09:02.382 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2026-02-19T20:09:02.382 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2026-02-19T20:09:02.382 INFO:tasks.rbd_mirror_thrash:waiting for 3.6431139559414696 secs before reviving daemons 2026-02-19T20:09:02.383 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:09:02.382+0000 7fd21ab49640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 47950) UID: 1000 2026-02-19T20:09:02.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3941352448 bytes 2026-02-19T20:09:02.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3996808192 bytes 2026-02-19T20:09:02.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 4074713088 bytes 2026-02-19T20:09:02.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 4120885248 bytes 2026-02-19T20:09:02.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 4186563584 bytes 2026-02-19T20:09:02.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4220568576 bytes 2026-02-19T20:09:02.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4240748544 bytes 2026-02-19T20:09:02.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4319739904 bytes 2026-02-19T20:09:02.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4341254144 bytes 2026-02-19T20:09:02.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4387096576 bytes 2026-02-19T20:09:02.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4391432192 bytes 2026-02-19T20:09:02.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4451424256 bytes 2026-02-19T20:09:02.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4495602688 bytes 2026-02-19T20:09:02.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4527838208 bytes 2026-02-19T20:09:02.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4532581376 bytes 2026-02-19T20:09:03.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4585671680 bytes 2026-02-19T20:09:03.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4606413824 bytes 2026-02-19T20:09:03.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4607032320 bytes 2026-02-19T20:09:03.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4670294016 bytes 2026-02-19T20:09:03.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4718209024 bytes 2026-02-19T20:09:03.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4718824448 bytes 2026-02-19T20:09:03.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4778713088 bytes 2026-02-19T20:09:03.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4880692224 bytes 2026-02-19T20:09:03.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4910764032 bytes 2026-02-19T20:09:03.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4958204928 bytes 2026-02-19T20:09:03.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 5002678272 bytes 2026-02-19T20:09:03.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 5037562880 bytes 2026-02-19T20:09:03.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 5143074816 bytes 2026-02-19T20:09:03.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5143715840 bytes 2026-02-19T20:09:03.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5170999296 bytes 2026-02-19T20:09:03.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5241237504 bytes 2026-02-19T20:09:03.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5307040768 bytes 2026-02-19T20:09:03.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5355345920 bytes 2026-02-19T20:09:03.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5402823680 bytes 2026-02-19T20:09:03.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5455097856 bytes 2026-02-19T20:09:03.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5455713280 bytes 2026-02-19T20:09:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5496646656 bytes 2026-02-19T20:09:03.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5508345856 bytes 2026-02-19T20:09:04.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5608295424 bytes 2026-02-19T20:09:04.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5637694464 bytes 2026-02-19T20:09:04.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5638307840 bytes 2026-02-19T20:09:04.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5659663360 bytes 2026-02-19T20:09:04.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5700963328 bytes 2026-02-19T20:09:04.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5784198144 bytes 2026-02-19T20:09:04.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5842583552 bytes 2026-02-19T20:09:04.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5866711040 bytes 2026-02-19T20:09:04.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5889299456 bytes 2026-02-19T20:09:04.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5937362944 bytes 2026-02-19T20:09:04.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5991653376 bytes 2026-02-19T20:09:04.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 6082884608 bytes 2026-02-19T20:09:04.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 6146074624 bytes 2026-02-19T20:09:04.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 6196680704 bytes 2026-02-19T20:09:04.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6231035904 bytes 2026-02-19T20:09:04.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6277902336 bytes 2026-02-19T20:09:04.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6297654272 bytes 2026-02-19T20:09:04.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6300875776 bytes 2026-02-19T20:09:04.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6385539072 bytes 2026-02-19T20:09:04.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6391107584 bytes 2026-02-19T20:09:04.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6441135104 bytes 2026-02-19T20:09:04.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6494244864 bytes 2026-02-19T20:09:04.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6533161984 bytes 2026-02-19T20:09:04.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6592508928 bytes 2026-02-19T20:09:05.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6611098624 bytes 2026-02-19T20:09:05.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6637623296 bytes 2026-02-19T20:09:05.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6649284608 bytes 2026-02-19T20:09:05.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6666897408 bytes 2026-02-19T20:09:05.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6731681792 bytes 2026-02-19T20:09:05.230 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6765471744 bytes 2026-02-19T20:09:05.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6802190336 bytes 2026-02-19T20:09:05.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6817947648 bytes 2026-02-19T20:09:05.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6829711360 bytes 2026-02-19T20:09:05.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6871740416 bytes 2026-02-19T20:09:05.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6923138048 bytes 2026-02-19T20:09:05.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 7005451264 bytes 2026-02-19T20:09:05.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7044030464 bytes 2026-02-19T20:09:05.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7099991040 bytes 2026-02-19T20:09:05.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7118125056 bytes 2026-02-19T20:09:05.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7135928320 bytes 2026-02-19T20:09:05.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7241005056 bytes 2026-02-19T20:09:05.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7285251072 bytes 2026-02-19T20:09:05.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7339717632 bytes 2026-02-19T20:09:05.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7353226240 bytes 2026-02-19T20:09:05.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7430601728 bytes 2026-02-19T20:09:05.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7479204864 bytes 2026-02-19T20:09:05.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7548459008 bytes 2026-02-19T20:09:06.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7584435200 bytes 2026-02-19T20:09:06.026 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2026-02-19T20:09:06.026 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:09:06.026 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2026-02-19T20:09:06.026 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2026-02-19T20:09:06.026 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2026-02-19T20:09:06.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2026-02-19T20:09:06.029 INFO:tasks.rbd_mirror_thrash:waiting for 66.18782045590942 secs before thrashing 2026-02-19T20:09:06.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7613420544 bytes 2026-02-19T20:09:06.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7651713024 bytes 2026-02-19T20:09:06.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7711850496 bytes 2026-02-19T20:09:06.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7766648832 bytes 2026-02-19T20:09:06.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7790662656 bytes 2026-02-19T20:09:06.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7858102272 bytes 2026-02-19T20:09:06.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7910415360 bytes 2026-02-19T20:09:06.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7961362432 bytes 2026-02-19T20:09:06.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 8019695616 bytes 2026-02-19T20:09:06.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 8053309440 bytes 2026-02-19T20:09:06.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 8148067328 bytes 2026-02-19T20:09:06.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 8156895232 bytes 2026-02-19T20:09:06.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 8238703616 bytes 2026-02-19T20:09:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 8257345536 bytes 2026-02-19T20:09:06.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 8288674816 bytes 2026-02-19T20:09:06.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 8300414976 bytes 2026-02-19T20:09:06.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 8353306624 bytes 2026-02-19T20:09:06.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 8414452736 bytes 2026-02-19T20:09:06.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8460626944 bytes 2026-02-19T20:09:06.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8461239296 bytes 2026-02-19T20:09:06.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8549516288 bytes 2026-02-19T20:09:07.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8623845376 bytes 2026-02-19T20:09:07.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8672771072 bytes 2026-02-19T20:09:07.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8694407168 bytes 2026-02-19T20:09:07.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8715376640 bytes 2026-02-19T20:09:07.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8781785088 bytes 2026-02-19T20:09:07.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8789518336 bytes 2026-02-19T20:09:07.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8838369280 bytes 2026-02-19T20:09:07.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8867351552 bytes 2026-02-19T20:09:07.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8877740032 bytes 2026-02-19T20:09:07.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8911695872 bytes 2026-02-19T20:09:07.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8940442624 bytes 2026-02-19T20:09:07.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8981699584 bytes 2026-02-19T20:09:07.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8992566272 bytes 2026-02-19T20:09:07.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 9010339840 bytes 2026-02-19T20:09:07.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 9065023488 bytes 2026-02-19T20:09:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 9098783744 bytes 2026-02-19T20:09:07.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 9184616448 bytes 2026-02-19T20:09:07.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 9214483456 bytes 2026-02-19T20:09:07.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 9285137408 bytes 2026-02-19T20:09:07.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 9285736448 bytes 2026-02-19T20:09:07.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 9368433664 bytes 2026-02-19T20:09:07.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 9395678208 bytes 2026-02-19T20:09:07.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 9475511296 bytes 2026-02-19T20:09:07.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 9502789632 bytes 2026-02-19T20:09:07.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 9505448960 bytes 2026-02-19T20:09:08.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 9562536960 bytes 2026-02-19T20:09:08.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 9624650752 bytes 2026-02-19T20:09:08.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 9679173632 bytes 2026-02-19T20:09:08.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 9726603264 bytes 2026-02-19T20:09:08.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 9757176832 bytes 2026-02-19T20:09:08.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 9785436160 bytes 2026-02-19T20:09:08.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 9843706880 bytes 2026-02-19T20:09:08.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 9899251712 bytes 2026-02-19T20:09:08.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 9899879424 bytes 2026-02-19T20:09:08.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 9916808192 bytes 2026-02-19T20:09:08.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 9921846272 bytes 2026-02-19T20:09:08.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 9978835968 bytes 2026-02-19T20:09:08.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22600 IOs, 10005787648 bytes 2026-02-19T20:09:08.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22700 IOs, 10039662592 bytes 2026-02-19T20:09:08.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22800 IOs, 10054669312 bytes 2026-02-19T20:09:08.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22900 IOs, 10082274304 bytes 2026-02-19T20:09:08.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23000 IOs, 10144331776 bytes 2026-02-19T20:09:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23100 IOs, 10172407808 bytes 2026-02-19T20:09:08.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23200 IOs, 10189624320 bytes 2026-02-19T20:09:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23300 IOs, 10233390080 bytes 2026-02-19T20:09:08.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23400 IOs, 10255460352 bytes 2026-02-19T20:09:09.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23500 IOs, 10300121088 bytes 2026-02-19T20:09:09.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23600 IOs, 10339826688 bytes 2026-02-19T20:09:09.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23700 IOs, 10364829696 bytes 2026-02-19T20:09:09.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23800 IOs, 10392549376 bytes 2026-02-19T20:09:09.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23900 IOs, 10406333440 bytes 2026-02-19T20:09:09.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24000 IOs, 10418317312 bytes 2026-02-19T20:09:09.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24100 IOs, 10442361856 bytes 2026-02-19T20:09:09.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24200 IOs, 10485810176 bytes 2026-02-19T20:09:09.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24300 IOs, 10543656960 bytes 2026-02-19T20:09:09.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24400 IOs, 10578789376 bytes 2026-02-19T20:09:09.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24500 IOs, 10639242240 bytes 2026-02-19T20:09:09.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24600 IOs, 10668142592 bytes 2026-02-19T20:09:09.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24700 IOs, 10709749760 bytes 2026-02-19T20:09:09.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24800 IOs, 10836964352 bytes 2026-02-19T20:09:09.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:24900 IOs, 10876069888 bytes 2026-02-19T20:09:09.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25000 IOs, 10919385088 bytes 2026-02-19T20:09:09.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25100 IOs, 10952087552 bytes 2026-02-19T20:09:09.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25200 IOs, 10985761792 bytes 2026-02-19T20:09:09.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25300 IOs, 11111772160 bytes 2026-02-19T20:09:09.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25400 IOs, 11156097024 bytes 2026-02-19T20:09:09.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25500 IOs, 11202813952 bytes 2026-02-19T20:09:09.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25600 IOs, 11245181952 bytes 2026-02-19T20:09:10.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25700 IOs, 11316762624 bytes 2026-02-19T20:09:10.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25800 IOs, 11331033088 bytes 2026-02-19T20:09:10.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:25900 IOs, 11360342016 bytes 2026-02-19T20:09:10.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26000 IOs, 11366587392 bytes 2026-02-19T20:09:10.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26100 IOs, 11382553600 bytes 2026-02-19T20:09:10.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26200 IOs, 11405191168 bytes 2026-02-19T20:09:10.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26300 IOs, 11471279104 bytes 2026-02-19T20:09:10.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26400 IOs, 11521715200 bytes 2026-02-19T20:09:10.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26500 IOs, 11522343936 bytes 2026-02-19T20:09:10.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26600 IOs, 11522952192 bytes 2026-02-19T20:09:10.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26700 IOs, 11544951808 bytes 2026-02-19T20:09:10.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26800 IOs, 11622473728 bytes 2026-02-19T20:09:10.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:26900 IOs, 11671212032 bytes 2026-02-19T20:09:10.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27000 IOs, 11681301504 bytes 2026-02-19T20:09:10.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27100 IOs, 11720904704 bytes 2026-02-19T20:09:10.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27200 IOs, 11721509888 bytes 2026-02-19T20:09:10.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27300 IOs, 11756025856 bytes 2026-02-19T20:09:10.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27400 IOs, 11789058048 bytes 2026-02-19T20:09:10.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27500 IOs, 11807042560 bytes 2026-02-19T20:09:10.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27600 IOs, 11876561920 bytes 2026-02-19T20:09:10.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27700 IOs, 11903441920 bytes 2026-02-19T20:09:10.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27800 IOs, 11955921920 bytes 2026-02-19T20:09:10.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:27900 IOs, 11972776960 bytes 2026-02-19T20:09:10.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28000 IOs, 12043203584 bytes 2026-02-19T20:09:10.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28100 IOs, 12098774016 bytes 2026-02-19T20:09:10.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28200 IOs, 12118325248 bytes 2026-02-19T20:09:11.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28300 IOs, 12182418432 bytes 2026-02-19T20:09:11.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28400 IOs, 12228505600 bytes 2026-02-19T20:09:11.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28500 IOs, 12334176256 bytes 2026-02-19T20:09:11.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28600 IOs, 12391087104 bytes 2026-02-19T20:09:11.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28700 IOs, 12443574272 bytes 2026-02-19T20:09:11.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28800 IOs, 12486128640 bytes 2026-02-19T20:09:11.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:28900 IOs, 12514125824 bytes 2026-02-19T20:09:11.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29000 IOs, 12563490816 bytes 2026-02-19T20:09:11.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29100 IOs, 12607133696 bytes 2026-02-19T20:09:11.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29200 IOs, 12632718336 bytes 2026-02-19T20:09:11.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29300 IOs, 12667670528 bytes 2026-02-19T20:09:11.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29400 IOs, 12730145792 bytes 2026-02-19T20:09:11.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29500 IOs, 12790949888 bytes 2026-02-19T20:09:11.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29600 IOs, 12863226880 bytes 2026-02-19T20:09:11.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29700 IOs, 12950557696 bytes 2026-02-19T20:09:11.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29800 IOs, 13004228608 bytes 2026-02-19T20:09:11.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:29900 IOs, 13050898432 bytes 2026-02-19T20:09:11.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30000 IOs, 13100358656 bytes 2026-02-19T20:09:11.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30100 IOs, 13124851712 bytes 2026-02-19T20:09:12.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30200 IOs, 13196171264 bytes 2026-02-19T20:09:12.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30300 IOs, 13254708224 bytes 2026-02-19T20:09:12.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30400 IOs, 13272876032 bytes 2026-02-19T20:09:12.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30500 IOs, 13344646144 bytes 2026-02-19T20:09:12.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30600 IOs, 13378386944 bytes 2026-02-19T20:09:12.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30700 IOs, 13395980288 bytes 2026-02-19T20:09:12.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30800 IOs, 13414189056 bytes 2026-02-19T20:09:12.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:30900 IOs, 13482216448 bytes 2026-02-19T20:09:12.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:31000 IOs, 13529288704 bytes 2026-02-19T20:09:12.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:31100 IOs, 13554036736 bytes 2026-02-19T20:09:12.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:31200 IOs, 13608023040 bytes 2026-02-19T20:09:12.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:31300 IOs, 13629495296 bytes 2026-02-19T20:09:12.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:31400 IOs, 13636728832 bytes 2026-02-19T20:09:12.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:31500 IOs, 13701027840 bytes 2026-02-19T20:09:12.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:31600 IOs, 13734925312 bytes 2026-02-19T20:09:12.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:31700 IOs, 13770394624 bytes 2026-02-19T20:09:12.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:31800 IOs, 13771889664 bytes 2026-02-19T20:09:12.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:09:12.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:09:12.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:09:13.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:09:13.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:09:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:09:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:09:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:09:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:09:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:09:13.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:09:13.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:09:13.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:09:13.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"last_snapshot_bytes":0,"last_snapshot_sync_seconds":0,"remote_snapshot_timestamp":1771531713,"replay_state":"idle"} 2026-02-19T20:09:13.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:09:13.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:08:58 2026-02-19T20:09:13.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:09:13.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:09:13.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: down+unknown 2026-02-19T20:09:13.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: status not found 2026-02-19T20:09:13.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: ' 2026-02-19T20:09:13.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:09:13.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:09:13.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:09:13.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"last_snapshot_bytes":0,"last_snapshot_sync_seconds":0,"remote_snapshot_timestamp":1771531713,"replay_state":"idle"} 2026-02-19T20:09:13.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:09:13.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:09:13.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:09:13.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:09:13.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:09:13.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:09:13.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:09:13.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:09:13.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:08:58' 2026-02-19T20:09:13.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:08:58' 2026-02-19T20:09:13.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:13.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:13.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:13.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:09:13.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:08:58' 2026-02-19T20:09:13.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:09:13.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:09:13.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:09:13.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='WARNINGOKWARNING142024288mirror.0trial08020.3.0-5336-g802bf111falseOK42054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK42124289mirror.3trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"last_snapshot_bytes":0,"last_snapshot_sync_seconds":0,"remote_snapshot_timestamp":1771531713,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:08:58cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3down+unknownstatus not found' 2026-02-19T20:09:13.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:09:13.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:09:13.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:08:58' 2026-02-19T20:09:13.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:09:13.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:09:13.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:09:13.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap1 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap1 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap1 2026-02-19T20:09:13.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:09:26.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9633790395964058) > weight (0.25) 2026-02-19T20:09:26.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5100955146487244) > weight (0.25) 2026-02-19T20:09:26.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.30264483790702845) > weight (0.25) 2026-02-19T20:09:26.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5249337089027175) > weight (0.25) 2026-02-19T20:09:26.057 INFO:tasks.rbd_mirror_thrash:waiting for 75.2980713964947 secs before thrashing 2026-02-19T20:09:43.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:09:43.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:43.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:43.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:43.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:09:43.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:43.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:43.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:43.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:09:43.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:09:43.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:09:43.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:09:44.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:09:44.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:44.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:44.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:44.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:09:44.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:09:44.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:09:44.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:09:44.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:09:44.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:09:44.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:09:44.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:09:44.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:09:44.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:09:44.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:09:44.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:09:44.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:09:44.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:09:44.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:09:44.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:09:44.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:09:44.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:09:44.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:09:44.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:09:44.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:09:44.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:09:44.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:09:44.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:09:44.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:09:44.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:09:44.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:09:44.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:09:44.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:09:44.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:09:44.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:09:44.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:09:44.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:09:44.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:09:44.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:09:44.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:09:44.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:09:44.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:09:44.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:09:44.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:09:44.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:09:44.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:09:44.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:09:44.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:09:44.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:09:44.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:09:44.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:09:44.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:09:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:09:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:09:44.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:09:44.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:09:44.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:09:44.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:09:44.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:09:44.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:09:44.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:09:44.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:09:44.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:09:44.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:09:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:09:45.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 5 2026-02-19T20:09:45.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:09:45.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:09:45.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:09:45.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:09:45.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:09:45.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:45.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:09:45.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:09:45.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:09:45.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:09:45.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:09:45.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:09:45.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:09:45.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:09:45.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:09:45.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:09:45.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:45.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:45.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:09:45.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:09:45.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:09:45.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:09:45.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=5 2026-02-19T20:09:45.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:09:45.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:09:45.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:09:45.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:09:45.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:45.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:45.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:45.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:09:45.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:09:45.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:09:45.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:09:45.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=3 2026-02-19T20:09:45.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 5 = 3 2026-02-19T20:09:45.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:09:45.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:09:46.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:09:46.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:46.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:46.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:46.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:09:46.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:09:46.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:09:46.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:09:46.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=3 2026-02-19T20:09:46.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 5 = 3 2026-02-19T20:09:46.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:09:46.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:09:46.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:09:46.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:46.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:46.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:46.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:09:46.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:09:46.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:09:46.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:09:46.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:09:46.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:09:46.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:09:46.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:09:46.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:09:46.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 5 = '' 2026-02-19T20:09:46.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:09:46.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:09:48.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:09:48.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:48.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:48.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:48.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:09:48.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:09:48.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:09:48.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:09:48.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=5 2026-02-19T20:09:48.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 5 = 5 2026-02-19T20:09:48.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:48.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap1 2026-02-19T20:09:48.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:48.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:48.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:48.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap1 2026-02-19T20:09:48.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:09:48.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:09:48.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:09:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap1 2026-02-19T20:09:49.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:09:49.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:09:49.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:09:49.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:09:49.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:09:49.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:09:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:09:49.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:49.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:09:49.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap1 2026-02-19T20:09:49.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:49.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:49.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap1 2026-02-19T20:09:49.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:09:49.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:09:49.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:09:49.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:09:49.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:09:50.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:09:50.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:09:50.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:09:50.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:09:50.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:09:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap1-clean 2026-02-19T20:09:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap1-clean 2026-02-19T20:09:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:09:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap1-clean 2026-02-19T20:09:50.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:09:51.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:09:51.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:09:51.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:09:51.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:51.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:51.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:09:51.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=5 2026-02-19T20:09:51.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:09:51.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:09:51.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 45551616 bytes 2026-02-19T20:09:51.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 46169088 bytes 2026-02-19T20:09:51.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 58440704 bytes 2026-02-19T20:09:51.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 71490560 bytes 2026-02-19T20:09:51.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 99661824 bytes 2026-02-19T20:09:51.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 150583296 bytes 2026-02-19T20:09:51.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 191674368 bytes 2026-02-19T20:09:51.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 216640512 bytes 2026-02-19T20:09:51.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 226153472 bytes 2026-02-19T20:09:51.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 315792384 bytes 2026-02-19T20:09:51.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 343744512 bytes 2026-02-19T20:09:52.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 390607872 bytes 2026-02-19T20:09:52.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 411674624 bytes 2026-02-19T20:09:52.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 481075200 bytes 2026-02-19T20:09:52.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 543693824 bytes 2026-02-19T20:09:52.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 583285760 bytes 2026-02-19T20:09:52.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 611985408 bytes 2026-02-19T20:09:52.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 660766720 bytes 2026-02-19T20:09:52.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 736621568 bytes 2026-02-19T20:09:52.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 758345728 bytes 2026-02-19T20:09:52.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 786184192 bytes 2026-02-19T20:09:52.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 800507904 bytes 2026-02-19T20:09:52.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 821920768 bytes 2026-02-19T20:09:52.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 836816896 bytes 2026-02-19T20:09:52.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 880476160 bytes 2026-02-19T20:09:52.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 928427008 bytes 2026-02-19T20:09:52.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1033735168 bytes 2026-02-19T20:09:52.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1089959936 bytes 2026-02-19T20:09:52.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1188772864 bytes 2026-02-19T20:09:52.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1213268992 bytes 2026-02-19T20:09:52.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1264104448 bytes 2026-02-19T20:09:52.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1280430080 bytes 2026-02-19T20:09:52.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1350247424 bytes 2026-02-19T20:09:53.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1397368832 bytes 2026-02-19T20:09:53.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1487612928 bytes 2026-02-19T20:09:53.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1516573696 bytes 2026-02-19T20:09:53.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1566319616 bytes 2026-02-19T20:09:53.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1633665024 bytes 2026-02-19T20:09:53.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1657378816 bytes 2026-02-19T20:09:53.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1697138688 bytes 2026-02-19T20:09:53.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1750680576 bytes 2026-02-19T20:09:53.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1803450368 bytes 2026-02-19T20:09:53.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1805574144 bytes 2026-02-19T20:09:53.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1896223744 bytes 2026-02-19T20:09:53.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1928369152 bytes 2026-02-19T20:09:53.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1941901312 bytes 2026-02-19T20:09:53.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2031585280 bytes 2026-02-19T20:09:53.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2115107840 bytes 2026-02-19T20:09:53.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2172618752 bytes 2026-02-19T20:09:53.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2205658112 bytes 2026-02-19T20:09:53.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2263948288 bytes 2026-02-19T20:09:54.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2316725248 bytes 2026-02-19T20:09:54.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2390142976 bytes 2026-02-19T20:09:54.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2429485056 bytes 2026-02-19T20:09:54.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2499709952 bytes 2026-02-19T20:09:54.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2508073984 bytes 2026-02-19T20:09:54.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2550247424 bytes 2026-02-19T20:09:54.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2604946432 bytes 2026-02-19T20:09:54.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2640434176 bytes 2026-02-19T20:09:54.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2658369536 bytes 2026-02-19T20:09:54.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2710067200 bytes 2026-02-19T20:09:54.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2753072128 bytes 2026-02-19T20:09:54.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2771784704 bytes 2026-02-19T20:09:54.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2885601280 bytes 2026-02-19T20:09:54.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2930776064 bytes 2026-02-19T20:09:54.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2982892544 bytes 2026-02-19T20:09:54.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3014740992 bytes 2026-02-19T20:09:54.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3039496192 bytes 2026-02-19T20:09:54.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3040113664 bytes 2026-02-19T20:09:54.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3055691776 bytes 2026-02-19T20:09:55.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3092005888 bytes 2026-02-19T20:09:55.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3094222848 bytes 2026-02-19T20:09:55.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3160127488 bytes 2026-02-19T20:09:55.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3197691904 bytes 2026-02-19T20:09:55.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3236602880 bytes 2026-02-19T20:09:55.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3251278848 bytes 2026-02-19T20:09:55.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3299236864 bytes 2026-02-19T20:09:55.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3350765568 bytes 2026-02-19T20:09:55.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3380919296 bytes 2026-02-19T20:09:55.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3415655424 bytes 2026-02-19T20:09:55.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3463550976 bytes 2026-02-19T20:09:55.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3550918656 bytes 2026-02-19T20:09:55.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3618368512 bytes 2026-02-19T20:09:55.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3629981696 bytes 2026-02-19T20:09:55.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3667869696 bytes 2026-02-19T20:09:55.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3712758784 bytes 2026-02-19T20:09:55.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3730091008 bytes 2026-02-19T20:09:55.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3770502144 bytes 2026-02-19T20:09:55.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3847476224 bytes 2026-02-19T20:09:56.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3858463744 bytes 2026-02-19T20:09:56.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3900686336 bytes 2026-02-19T20:09:56.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3970350080 bytes 2026-02-19T20:09:56.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4003461120 bytes 2026-02-19T20:09:56.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4063015936 bytes 2026-02-19T20:09:56.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4098603008 bytes 2026-02-19T20:09:56.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4135543808 bytes 2026-02-19T20:09:56.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4176465920 bytes 2026-02-19T20:09:56.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4215422976 bytes 2026-02-19T20:09:56.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:09:56.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:09:56.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:09:56.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:56.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:09:56.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:56.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:56.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:56.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:09:56.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:09:56.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:09:56.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:09:56.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:09:57.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:09:57.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:57.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:57.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:57.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:09:57.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:09:57.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:09:57.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:09:57.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:09:57.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:09:57.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":17888426.67,"bytes_per_snapshot":268326400.0,"last_snapshot_bytes":536652800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531785,"remote_snapshot_timestamp":1771531785,"replay_state":"idle"} 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:09:56 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+stopped 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: local image is primary 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:09:56' 2026-02-19T20:09:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:09:57.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:09:57.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:09:57.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":17888426.67,"bytes_per_snapshot":268326400.0,"last_snapshot_bytes":536652800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531785,"remote_snapshot_timestamp":1771531785,"replay_state":"idle"} 2026-02-19T20:09:57.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:09:57.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:09:57.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:09:57.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:09:57.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:09:57.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:09:57.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:09:57.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:09:57.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:09:56' 2026-02-19T20:09:57.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:09:56' 2026-02-19T20:09:57.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:09:57.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:57.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:57.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:09:57.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:09:56' 2026-02-19T20:09:57.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:09:57.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:09:57.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:09:57.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='OKOKOK142024288mirror.0trial08020.3.0-5336-g802bf111falseOK42054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK42124289mirror.3trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":17888426.67,"bytes_per_snapshot":268326400.0,"last_snapshot_bytes":536652800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531785,"remote_snapshot_timestamp":1771531785,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:09:56cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3up+stoppedlocal image is primary2026-02-19 20:09:56' 2026-02-19T20:09:57.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:09:57.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:09:57.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:09:56' 2026-02-19T20:09:57.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:09:57.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:09:57.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:09:57.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:09:57.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:09:57.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:57.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:09:57.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap2 2026-02-19T20:09:57.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap2 2026-02-19T20:09:57.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:09:57.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:09:57.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:09:57.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap2 2026-02-19T20:09:57.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:10:12.217 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5668102151410708) > weight (0.25) 2026-02-19T20:10:12.217 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6590961668815992) > weight (0.25) 2026-02-19T20:10:12.217 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9038593136619968) > weight (0.25) 2026-02-19T20:10:12.217 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6952888914340126) > weight (0.25) 2026-02-19T20:10:12.217 INFO:tasks.rbd_mirror_thrash:waiting for 117.11781920413833 secs before thrashing 2026-02-19T20:10:26.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:10:26.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:26.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:26.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:26.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:10:26.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:26.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:26.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:26.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:10:26.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:10:26.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:10:26.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:10:27.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:10:27.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:10:27.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:10:27.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:10:27.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:10:27.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:10:27.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:10:27.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:10:27.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:10:27.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:10:27.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:10:27.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:10:27.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:10:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:10:27.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:10:27.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:10:27.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:10:27.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:10:27.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:10:27.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:10:27.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:10:27.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:10:27.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:10:27.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:10:27.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:10:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:10:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:10:27.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:10:27.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:10:27.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:10:27.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:10:27.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:10:27.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:10:27.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:10:27.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:10:27.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:10:27.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:10:27.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:10:27.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:10:27.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:10:27.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:10:27.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:10:27.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:10:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:10:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:10:27.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:10:27.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:10:27.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:10:27.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:10:27.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:10:27.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:10:27.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:10:27.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:10:27.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:10:27.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:10:27.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:10:27.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:10:27.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:10:27.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:10:27.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:10:27.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:10:27.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:10:27.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:10:27.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:27.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:10:27.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:10:27.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:10:27.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:10:27.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:27.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:27.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:10:29.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 8 2026-02-19T20:10:29.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:10:29.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:10:29.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:10:29.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:10:29.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:10:29.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:29.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:10:29.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:10:29.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:10:29.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:10:29.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:10:29.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:10:29.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:10:29.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:10:29.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:10:29.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:10:29.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:29.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:29.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:10:29.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:10:29.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:10:29.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:10:29.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=8 2026-02-19T20:10:29.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:10:29.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:10:29.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:10:29.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:10:29.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:29.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:29.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:29.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:10:29.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:10:29.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:10:29.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:10:29.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:10:29.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:10:29.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:10:29.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:10:29.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:10:29.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 8 = '' 2026-02-19T20:10:29.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:10:29.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:10:30.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:10:30.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:30.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:30.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:30.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:10:30.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:10:30.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:10:30.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:10:30.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:10:30.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:10:30.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:10:30.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:10:30.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:10:30.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 8 = '' 2026-02-19T20:10:30.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:10:30.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:10:30.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:10:30.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:30.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:30.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:30.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:10:30.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:10:30.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:10:31.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:10:31.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:10:31.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:10:31.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:10:31.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:10:31.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:10:31.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 8 = '' 2026-02-19T20:10:31.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:10:31.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:10:32.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:10:32.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:32.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:32.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:32.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:10:32.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:10:32.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:10:32.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:10:32.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=8 2026-02-19T20:10:32.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 8 = 8 2026-02-19T20:10:32.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:10:32.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap2 2026-02-19T20:10:32.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:32.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:32.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:32.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap2 2026-02-19T20:10:32.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:10:32.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:10:32.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:10:33.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap2 2026-02-19T20:10:33.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:10:33.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:10:33.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 4 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:10:33.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snap1-clean ']' 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap1-clean 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap1-clean 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:33.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:33.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:10:34.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:10:34.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:34.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:10:34.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:34.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:34.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:10:34.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap2 2026-02-19T20:10:34.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:34.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:34.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap2 2026-02-19T20:10:34.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:10:34.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:10:34.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:34.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:34.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:10:35.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:10:35.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:35.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:10:35.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:35.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:10:35.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:10:35.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap2-clean 2026-02-19T20:10:35.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap2-clean 2026-02-19T20:10:35.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:10:35.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:35.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:36.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap2-clean 2026-02-19T20:10:36.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:10:36.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:10:36.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:10:36.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:10:36.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:36.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:36.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:10:36.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=9 2026-02-19T20:10:36.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:10:36.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:10:36.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 6413312 bytes 2026-02-19T20:10:36.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 112323584 bytes 2026-02-19T20:10:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 150872064 bytes 2026-02-19T20:10:36.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 222490624 bytes 2026-02-19T20:10:36.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 281266176 bytes 2026-02-19T20:10:36.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 302817280 bytes 2026-02-19T20:10:36.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 336109568 bytes 2026-02-19T20:10:36.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 370465792 bytes 2026-02-19T20:10:37.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 423053312 bytes 2026-02-19T20:10:37.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 446664704 bytes 2026-02-19T20:10:37.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 481747968 bytes 2026-02-19T20:10:37.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 525258752 bytes 2026-02-19T20:10:37.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 590035968 bytes 2026-02-19T20:10:37.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 620136448 bytes 2026-02-19T20:10:37.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 660135936 bytes 2026-02-19T20:10:37.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 754543616 bytes 2026-02-19T20:10:37.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 780887040 bytes 2026-02-19T20:10:37.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 827640832 bytes 2026-02-19T20:10:37.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 871101440 bytes 2026-02-19T20:10:37.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 909584384 bytes 2026-02-19T20:10:37.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 933249024 bytes 2026-02-19T20:10:37.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 978576384 bytes 2026-02-19T20:10:37.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1002416128 bytes 2026-02-19T20:10:37.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1044909056 bytes 2026-02-19T20:10:37.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1101027328 bytes 2026-02-19T20:10:37.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1150672896 bytes 2026-02-19T20:10:37.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1192910848 bytes 2026-02-19T20:10:37.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1231138816 bytes 2026-02-19T20:10:37.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1254017024 bytes 2026-02-19T20:10:37.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1294290944 bytes 2026-02-19T20:10:38.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1297267712 bytes 2026-02-19T20:10:38.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1367140352 bytes 2026-02-19T20:10:38.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1440248832 bytes 2026-02-19T20:10:38.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1494695936 bytes 2026-02-19T20:10:38.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1543315456 bytes 2026-02-19T20:10:38.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1577194496 bytes 2026-02-19T20:10:38.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1627677696 bytes 2026-02-19T20:10:38.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1674280960 bytes 2026-02-19T20:10:38.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1709797376 bytes 2026-02-19T20:10:38.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1796429824 bytes 2026-02-19T20:10:38.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1817081856 bytes 2026-02-19T20:10:38.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1829345280 bytes 2026-02-19T20:10:38.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1864146944 bytes 2026-02-19T20:10:38.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1930105856 bytes 2026-02-19T20:10:38.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2064793600 bytes 2026-02-19T20:10:38.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2118811648 bytes 2026-02-19T20:10:38.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2165952512 bytes 2026-02-19T20:10:38.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2232833024 bytes 2026-02-19T20:10:38.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2233427968 bytes 2026-02-19T20:10:39.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2270701568 bytes 2026-02-19T20:10:39.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2284441600 bytes 2026-02-19T20:10:39.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2333946880 bytes 2026-02-19T20:10:39.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2347310080 bytes 2026-02-19T20:10:39.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2397456384 bytes 2026-02-19T20:10:39.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2459697152 bytes 2026-02-19T20:10:39.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2519417856 bytes 2026-02-19T20:10:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2567378944 bytes 2026-02-19T20:10:39.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2596100096 bytes 2026-02-19T20:10:39.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2662413312 bytes 2026-02-19T20:10:39.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2663030784 bytes 2026-02-19T20:10:39.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2748624896 bytes 2026-02-19T20:10:39.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2801129472 bytes 2026-02-19T20:10:39.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2889385984 bytes 2026-02-19T20:10:39.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2946665472 bytes 2026-02-19T20:10:39.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3020310528 bytes 2026-02-19T20:10:39.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3081540608 bytes 2026-02-19T20:10:39.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3141612544 bytes 2026-02-19T20:10:39.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3182921728 bytes 2026-02-19T20:10:39.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3228238848 bytes 2026-02-19T20:10:40.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3270455296 bytes 2026-02-19T20:10:40.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3307283456 bytes 2026-02-19T20:10:40.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3368205312 bytes 2026-02-19T20:10:40.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3368819712 bytes 2026-02-19T20:10:40.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3415770112 bytes 2026-02-19T20:10:40.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3457540096 bytes 2026-02-19T20:10:40.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3504371712 bytes 2026-02-19T20:10:40.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3538003968 bytes 2026-02-19T20:10:40.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3582744576 bytes 2026-02-19T20:10:40.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3615823872 bytes 2026-02-19T20:10:40.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3643200512 bytes 2026-02-19T20:10:40.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3666156544 bytes 2026-02-19T20:10:40.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3714359296 bytes 2026-02-19T20:10:40.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3819184128 bytes 2026-02-19T20:10:40.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3885011968 bytes 2026-02-19T20:10:40.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3938288640 bytes 2026-02-19T20:10:40.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3938917376 bytes 2026-02-19T20:10:40.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4039785472 bytes 2026-02-19T20:10:40.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4102203392 bytes 2026-02-19T20:10:40.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4133478400 bytes 2026-02-19T20:10:40.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4177313792 bytes 2026-02-19T20:10:40.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4192983040 bytes 2026-02-19T20:10:41.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4283205632 bytes 2026-02-19T20:10:41.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4314832896 bytes 2026-02-19T20:10:41.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4352168960 bytes 2026-02-19T20:10:41.196 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4391979008 bytes 2026-02-19T20:10:41.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4439823360 bytes 2026-02-19T20:10:41.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4500011008 bytes 2026-02-19T20:10:41.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8607269705638139) > weight (0.25) 2026-02-19T20:10:41.357 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8618378723074119) > weight (0.25) 2026-02-19T20:10:41.357 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5329871188583032) > weight (0.25) 2026-02-19T20:10:41.357 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.79310067114328) > weight (0.25) 2026-02-19T20:10:41.357 INFO:tasks.rbd_mirror_thrash:waiting for 72.1600156586755 secs before thrashing 2026-02-19T20:10:41.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4547638272 bytes 2026-02-19T20:10:41.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4577314816 bytes 2026-02-19T20:10:41.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4620991488 bytes 2026-02-19T20:10:41.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4682132480 bytes 2026-02-19T20:10:41.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4746536960 bytes 2026-02-19T20:10:41.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4756542464 bytes 2026-02-19T20:10:41.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4806837248 bytes 2026-02-19T20:10:41.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4809695232 bytes 2026-02-19T20:10:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4867858432 bytes 2026-02-19T20:10:41.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4930284544 bytes 2026-02-19T20:10:41.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4955045888 bytes 2026-02-19T20:10:41.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4992905216 bytes 2026-02-19T20:10:41.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5058562048 bytes 2026-02-19T20:10:41.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5103639552 bytes 2026-02-19T20:10:42.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5161541632 bytes 2026-02-19T20:10:42.083 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5195318272 bytes 2026-02-19T20:10:42.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5239400448 bytes 2026-02-19T20:10:42.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5262574592 bytes 2026-02-19T20:10:42.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5281195008 bytes 2026-02-19T20:10:42.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5290801152 bytes 2026-02-19T20:10:42.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5310655488 bytes 2026-02-19T20:10:42.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5425825792 bytes 2026-02-19T20:10:42.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5475252224 bytes 2026-02-19T20:10:42.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5529623552 bytes 2026-02-19T20:10:42.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5594058752 bytes 2026-02-19T20:10:42.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5631905792 bytes 2026-02-19T20:10:42.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5705551872 bytes 2026-02-19T20:10:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5742168064 bytes 2026-02-19T20:10:42.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5779032064 bytes 2026-02-19T20:10:43.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5878876160 bytes 2026-02-19T20:10:43.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5912939520 bytes 2026-02-19T20:10:43.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5935640576 bytes 2026-02-19T20:10:43.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5962177536 bytes 2026-02-19T20:10:43.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5980782592 bytes 2026-02-19T20:10:43.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 6052954112 bytes 2026-02-19T20:10:43.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 6104845312 bytes 2026-02-19T20:10:43.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6126028800 bytes 2026-02-19T20:10:43.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6185639936 bytes 2026-02-19T20:10:43.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6247321600 bytes 2026-02-19T20:10:43.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6353244160 bytes 2026-02-19T20:10:43.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6367327232 bytes 2026-02-19T20:10:43.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6454895616 bytes 2026-02-19T20:10:43.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6476475392 bytes 2026-02-19T20:10:43.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6535498752 bytes 2026-02-19T20:10:43.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6570460160 bytes 2026-02-19T20:10:43.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6624931840 bytes 2026-02-19T20:10:43.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6730563584 bytes 2026-02-19T20:10:43.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6813330432 bytes 2026-02-19T20:10:44.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6869615616 bytes 2026-02-19T20:10:44.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6906000384 bytes 2026-02-19T20:10:44.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6926821376 bytes 2026-02-19T20:10:44.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6990814208 bytes 2026-02-19T20:10:44.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 7037597696 bytes 2026-02-19T20:10:44.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 7072702464 bytes 2026-02-19T20:10:44.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 7073298432 bytes 2026-02-19T20:10:44.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 7084247040 bytes 2026-02-19T20:10:44.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 7127665664 bytes 2026-02-19T20:10:44.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 7152942080 bytes 2026-02-19T20:10:44.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7181391872 bytes 2026-02-19T20:10:44.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7191754752 bytes 2026-02-19T20:10:44.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7211751424 bytes 2026-02-19T20:10:44.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7240762368 bytes 2026-02-19T20:10:44.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7280726016 bytes 2026-02-19T20:10:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7309534208 bytes 2026-02-19T20:10:44.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7320462336 bytes 2026-02-19T20:10:44.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7370242048 bytes 2026-02-19T20:10:44.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7373044736 bytes 2026-02-19T20:10:44.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7378397184 bytes 2026-02-19T20:10:44.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7439558656 bytes 2026-02-19T20:10:44.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7568300032 bytes 2026-02-19T20:10:44.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7568919552 bytes 2026-02-19T20:10:44.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7586224128 bytes 2026-02-19T20:10:44.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7637060608 bytes 2026-02-19T20:10:45.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7650658304 bytes 2026-02-19T20:10:45.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7714843648 bytes 2026-02-19T20:10:45.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7743258624 bytes 2026-02-19T20:10:45.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7797000192 bytes 2026-02-19T20:10:45.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7797609472 bytes 2026-02-19T20:10:45.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7847694336 bytes 2026-02-19T20:10:45.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7946242048 bytes 2026-02-19T20:10:45.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 8011796480 bytes 2026-02-19T20:10:45.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 8071554048 bytes 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:10:45.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:10:46.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:10:46.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:46.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:46.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:46.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:10:46.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:10:46.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:10:46.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:10:46.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:10:46.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:10:46.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:10:46.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:10:46.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:10:46.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":268326400.0,"last_snapshot_bytes":536652800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531785,"remote_snapshot_timestamp":1771531785,"replay_state":"idle"} 2026-02-19T20:10:46.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:10:46.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:10:26 2026-02-19T20:10:46.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:10:46.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:10:46.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+stopped 2026-02-19T20:10:46.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: local image is primary 2026-02-19T20:10:46.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:10:26' 2026-02-19T20:10:46.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:10:46.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:10:46.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:10:46.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":268326400.0,"last_snapshot_bytes":536652800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531785,"remote_snapshot_timestamp":1771531785,"replay_state":"idle"} 2026-02-19T20:10:46.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:10:46.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:10:46.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:10:46.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:10:46.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:10:46.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:10:46.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:10:46.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:10:46.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:10:26' 2026-02-19T20:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:10:26' 2026-02-19T20:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:10:26' 2026-02-19T20:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:10:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:10:46.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='OKOKOK142024288mirror.0trial08020.3.0-5336-g802bf111falseOK42054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK42124289mirror.3trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":268326400.0,"last_snapshot_bytes":536652800,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531785,"remote_snapshot_timestamp":1771531785,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:10:26cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3up+stoppedlocal image is primary2026-02-19 20:10:26' 2026-02-19T20:10:46.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:10:46.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:10:46.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:10:26' 2026-02-19T20:10:46.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:10:46.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:10:46.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:10:46.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:10:46.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:10:46.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:10:46.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:10:46.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap3 2026-02-19T20:10:46.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap3 2026-02-19T20:10:46.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:10:46.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:10:46.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:10:46.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap3 2026-02-19T20:10:46.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:11:13.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:11:13.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:13.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:13.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:13.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:11:13.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:13.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:13.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:13.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:11:13.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:11:13.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:11:13.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:11:14.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:11:14.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:14.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:14.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:14.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:11:14.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:11:14.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:11:14.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:11:14.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:11:14.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:11:14.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:11:14.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:11:14.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:11:14.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:11:14.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:11:14.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:11:14.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:11:14.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:11:14.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:11:14.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:11:14.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:11:14.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:11:14.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:11:14.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:11:14.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:11:14.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:11:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:11:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:11:14.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:11:14.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:11:14.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:11:14.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:11:14.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:11:14.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:11:14.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:11:14.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:11:14.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:11:14.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:11:14.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:11:14.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:11:14.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:11:14.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:11:14.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:11:14.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:11:14.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:11:14.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:11:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:11:14.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:11:14.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:11:14.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:11:14.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:11:14.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:11:14.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:11:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:11:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:11:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:11:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:11:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:11:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:11:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:11:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:11:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:11:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:11:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:11:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:11:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:11:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:11:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:11:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:11:16.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 11 2026-02-19T20:11:16.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:11:16.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:11:16.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:11:16.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:11:16.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:11:16.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:16.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:11:16.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:11:16.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:11:16.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:11:16.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:11:16.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:11:16.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:11:16.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:11:16.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:11:16.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:11:16.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:16.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:16.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:11:16.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:11:16.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:11:16.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:11:16.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=11 2026-02-19T20:11:16.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:11:16.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:11:16.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:11:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:11:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:11:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:11:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:11:16.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:11:16.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:11:16.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:11:16.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:11:16.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:11:16.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:11:16.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 11 = '' 2026-02-19T20:11:16.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:11:16.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:11:17.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:11:17.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:17.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:17.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:17.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:11:17.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:11:17.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:11:17.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:11:17.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:11:17.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:11:17.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:11:17.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:11:17.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:11:17.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 11 = '' 2026-02-19T20:11:17.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:11:17.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:11:18.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:11:18.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:18.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:18.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:18.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:11:18.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:11:18.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:11:18.056 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:11:18.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:11:18.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:11:18.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:11:18.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:11:18.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:11:18.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 11 = '' 2026-02-19T20:11:18.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:11:18.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:11:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:11:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:11:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:11:19.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:11:19.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:11:19.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=11 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 11 = 11 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap3 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap3 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:11:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:11:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap3 2026-02-19T20:11:20.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:11:20.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:11:20.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:11:20.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 6 2026-02-19T20:11:20.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:11:20.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:11:20.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:11:20.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:11:20.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:11:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:11:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:11:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:10:34 2026 2026-02-19T20:11:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:11:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:11:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:11:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:11:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:11:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:11:20.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:11:20.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snap2-clean ']' 2026-02-19T20:11:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap2-clean 2026-02-19T20:11:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap2-clean 2026-02-19T20:11:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:11:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:11:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:11:21.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:11:21.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:21.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:11:21.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:21.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:21.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:11:21.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap3 2026-02-19T20:11:21.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:21.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:21.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap3 2026-02-19T20:11:21.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:11:21.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:11:21.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:21.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:21.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:11:22.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:11:22.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:22.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:11:22.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:22.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:11:23.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:11:23.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap3-clean 2026-02-19T20:11:23.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap3-clean 2026-02-19T20:11:23.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:11:23.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:23.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:23.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap3-clean 2026-02-19T20:11:23.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:11:23.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:11:23.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:11:23.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:11:23.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:23.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:23.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:11:23.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=6 2026-02-19T20:11:23.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:11:23.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:11:23.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 40956928 bytes 2026-02-19T20:11:23.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 98712576 bytes 2026-02-19T20:11:23.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 128719872 bytes 2026-02-19T20:11:23.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 168269824 bytes 2026-02-19T20:11:23.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 218479616 bytes 2026-02-19T20:11:23.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 269448192 bytes 2026-02-19T20:11:23.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 305159168 bytes 2026-02-19T20:11:23.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 325087232 bytes 2026-02-19T20:11:23.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 349447168 bytes 2026-02-19T20:11:24.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 396459008 bytes 2026-02-19T20:11:24.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 488398848 bytes 2026-02-19T20:11:24.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 489955328 bytes 2026-02-19T20:11:24.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 518700032 bytes 2026-02-19T20:11:24.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 582122496 bytes 2026-02-19T20:11:24.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 607494144 bytes 2026-02-19T20:11:24.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 625106944 bytes 2026-02-19T20:11:24.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 632194048 bytes 2026-02-19T20:11:24.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 649973760 bytes 2026-02-19T20:11:24.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 716706816 bytes 2026-02-19T20:11:24.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 761988096 bytes 2026-02-19T20:11:24.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 784305152 bytes 2026-02-19T20:11:24.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 798897152 bytes 2026-02-19T20:11:24.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 889862144 bytes 2026-02-19T20:11:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 924306432 bytes 2026-02-19T20:11:24.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 939576320 bytes 2026-02-19T20:11:24.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 978211840 bytes 2026-02-19T20:11:24.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 996748288 bytes 2026-02-19T20:11:24.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1066939392 bytes 2026-02-19T20:11:24.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1088971776 bytes 2026-02-19T20:11:24.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1089596416 bytes 2026-02-19T20:11:24.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1113643008 bytes 2026-02-19T20:11:24.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1152789504 bytes 2026-02-19T20:11:25.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1189646336 bytes 2026-02-19T20:11:25.083 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1223650304 bytes 2026-02-19T20:11:25.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1308945408 bytes 2026-02-19T20:11:25.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1340852224 bytes 2026-02-19T20:11:25.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1406899200 bytes 2026-02-19T20:11:25.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1452505088 bytes 2026-02-19T20:11:25.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1521230848 bytes 2026-02-19T20:11:25.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1534176256 bytes 2026-02-19T20:11:25.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1561900032 bytes 2026-02-19T20:11:25.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1586184192 bytes 2026-02-19T20:11:25.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1690932224 bytes 2026-02-19T20:11:25.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1769326592 bytes 2026-02-19T20:11:25.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1837597696 bytes 2026-02-19T20:11:25.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1861255168 bytes 2026-02-19T20:11:25.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1925189632 bytes 2026-02-19T20:11:25.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1929193472 bytes 2026-02-19T20:11:25.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1949871104 bytes 2026-02-19T20:11:25.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1968416768 bytes 2026-02-19T20:11:25.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2047762432 bytes 2026-02-19T20:11:26.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2107689984 bytes 2026-02-19T20:11:26.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2146769920 bytes 2026-02-19T20:11:26.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2216850432 bytes 2026-02-19T20:11:26.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2258610176 bytes 2026-02-19T20:11:26.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2313345024 bytes 2026-02-19T20:11:26.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2350467072 bytes 2026-02-19T20:11:26.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2372174848 bytes 2026-02-19T20:11:26.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2386604032 bytes 2026-02-19T20:11:26.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2446169088 bytes 2026-02-19T20:11:26.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2510100480 bytes 2026-02-19T20:11:26.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2559744000 bytes 2026-02-19T20:11:26.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2604266496 bytes 2026-02-19T20:11:26.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2639381504 bytes 2026-02-19T20:11:26.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2655894528 bytes 2026-02-19T20:11:26.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2708028416 bytes 2026-02-19T20:11:26.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2735600640 bytes 2026-02-19T20:11:26.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2750055424 bytes 2026-02-19T20:11:26.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2823902208 bytes 2026-02-19T20:11:26.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2891779072 bytes 2026-02-19T20:11:26.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2911335424 bytes 2026-02-19T20:11:27.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2986585088 bytes 2026-02-19T20:11:27.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3007352832 bytes 2026-02-19T20:11:27.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3026565120 bytes 2026-02-19T20:11:27.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3133118464 bytes 2026-02-19T20:11:27.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3168700416 bytes 2026-02-19T20:11:27.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3262169088 bytes 2026-02-19T20:11:27.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3269123072 bytes 2026-02-19T20:11:27.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3325651968 bytes 2026-02-19T20:11:27.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3338564608 bytes 2026-02-19T20:11:27.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3363802112 bytes 2026-02-19T20:11:27.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3403286528 bytes 2026-02-19T20:11:27.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3442266112 bytes 2026-02-19T20:11:27.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3484931072 bytes 2026-02-19T20:11:27.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3514608640 bytes 2026-02-19T20:11:27.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3575362560 bytes 2026-02-19T20:11:27.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3627795456 bytes 2026-02-19T20:11:27.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3641515008 bytes 2026-02-19T20:11:27.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3704607744 bytes 2026-02-19T20:11:27.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3712638976 bytes 2026-02-19T20:11:27.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3718023168 bytes 2026-02-19T20:11:27.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3765497856 bytes 2026-02-19T20:11:28.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3852165120 bytes 2026-02-19T20:11:28.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3921278976 bytes 2026-02-19T20:11:28.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3974963200 bytes 2026-02-19T20:11:28.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4035726336 bytes 2026-02-19T20:11:28.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4100814848 bytes 2026-02-19T20:11:28.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4108965888 bytes 2026-02-19T20:11:28.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4143290368 bytes 2026-02-19T20:11:28.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4230790144 bytes 2026-02-19T20:11:28.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4263781376 bytes 2026-02-19T20:11:28.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4290885632 bytes 2026-02-19T20:11:28.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4400223232 bytes 2026-02-19T20:11:28.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4451040256 bytes 2026-02-19T20:11:28.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4521029632 bytes 2026-02-19T20:11:28.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4548569088 bytes 2026-02-19T20:11:28.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4603019264 bytes 2026-02-19T20:11:28.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4632014848 bytes 2026-02-19T20:11:28.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4653685760 bytes 2026-02-19T20:11:28.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4708628480 bytes 2026-02-19T20:11:29.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4755462144 bytes 2026-02-19T20:11:29.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4829766656 bytes 2026-02-19T20:11:29.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4830355456 bytes 2026-02-19T20:11:29.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4942303232 bytes 2026-02-19T20:11:29.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4985250816 bytes 2026-02-19T20:11:29.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5009072128 bytes 2026-02-19T20:11:29.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5105667072 bytes 2026-02-19T20:11:29.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5150579712 bytes 2026-02-19T20:11:29.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5194058752 bytes 2026-02-19T20:11:29.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:11:29.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:11:29.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:11:29.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:11:29.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:11:29.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:29.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:29.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:29.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:11:29.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:11:29.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:11:29.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:11:29.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:11:30.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:11:30.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:30.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:30.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:30.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:11:30.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:11:30.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:11:30.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:11:30.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:11:30.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:11:30.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:11:30.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:11:30.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:11:30.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":17886958.93,"bytes_per_snapshot":536292352.0,"last_snapshot_bytes":536608768,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531875,"remote_snapshot_timestamp":1771531875,"replay_state":"idle"} 2026-02-19T20:11:30.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:11:30.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:11:26 2026-02-19T20:11:30.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:11:30.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:11:30.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+stopped 2026-02-19T20:11:30.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: local image is primary 2026-02-19T20:11:30.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:11:26' 2026-02-19T20:11:30.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:11:30.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:11:30.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:11:30.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":17886958.93,"bytes_per_snapshot":536292352.0,"last_snapshot_bytes":536608768,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531875,"remote_snapshot_timestamp":1771531875,"replay_state":"idle"} 2026-02-19T20:11:30.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:11:30.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:11:30.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:11:30.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:11:30.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:11:30.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:11:30.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:11:30.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:11:30.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:11:26' 2026-02-19T20:11:30.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:11:26' 2026-02-19T20:11:30.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:30.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:30.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:30.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:11:30.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:11:26' 2026-02-19T20:11:30.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:11:30.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:11:30.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:11:30.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='OKOKOK142024288mirror.0trial08020.3.0-5336-g802bf111falseOK42054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK42124289mirror.3trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":17886958.93,"bytes_per_snapshot":536292352.0,"last_snapshot_bytes":536608768,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531875,"remote_snapshot_timestamp":1771531875,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:11:26cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3up+stoppedlocal image is primary2026-02-19 20:11:26' 2026-02-19T20:11:30.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:11:30.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:11:30.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:11:26' 2026-02-19T20:11:30.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:11:30.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:11:30.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:11:30.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:11:30.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:11:30.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:11:30.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:11:30.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap4 2026-02-19T20:11:30.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap4 2026-02-19T20:11:30.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:11:30.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:30.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:30.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap4 2026-02-19T20:11:30.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:11:53.518 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6440225280497214) > weight (0.25) 2026-02-19T20:11:53.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.32977664283571384) > weight (0.25) 2026-02-19T20:11:53.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3455955180601704) > weight (0.25) 2026-02-19T20:11:53.519 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2026-02-19T20:11:53.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2026-02-19T20:11:53.519 INFO:tasks.rbd_mirror_thrash:waiting for 1.7038993792816337 secs before reviving daemons 2026-02-19T20:11:53.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:11:53.519+0000 7f1ec7462640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 47903) UID: 1000 2026-02-19T20:11:55.224 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2026-02-19T20:11:55.224 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:11:55.224 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2026-02-19T20:11:55.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2026-02-19T20:11:55.224 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2026-02-19T20:11:55.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2026-02-19T20:11:55.238 INFO:tasks.rbd_mirror_thrash:waiting for 85.8081803530468 secs before thrashing 2026-02-19T20:11:59.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:11:59.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:59.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:59.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:59.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:11:59.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:11:59.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:11:59.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:11:59.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:11:59.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:11:59.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:11:59.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:12:00.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:12:00.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:00.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:00.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:00.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:12:00.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:12:00.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:12:00.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:12:00.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:12:00.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:12:00.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:12:00.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:12:00.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:12:00.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:12:00.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:12:00.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:12:00.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:12:00.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:12:00.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:12:00.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:12:00.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:12:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:12:00.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:12:00.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:12:00.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:12:00.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:12:00.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:12:00.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:12:00.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:12:00.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:12:00.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:12:00.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:12:00.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:12:00.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:12:00.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:12:00.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:12:00.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:12:00.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:12:00.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:12:00.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:12:00.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:12:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:12:00.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:12:00.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:12:00.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:12:00.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:12:00.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:12:00.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:12:00.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:12:00.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:12:00.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:12:00.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:12:00.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:12:00.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:12:00.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:12:00.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:12:00.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:12:00.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:12:00.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:12:00.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:12:00.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:12:00.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:12:00.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:12:00.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:12:00.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:12:00.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:12:00.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:12:00.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:12:00.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:12:00.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:12:00.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:12:00.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:12:00.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:12:00.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:12:00.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:00.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:00.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:00.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:12:02.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 14 2026-02-19T20:12:02.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:12:02.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:12:02.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:02.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:12:02.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:12:02.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:02.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:12:02.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:12:02.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:12:02.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:12:02.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:12:02.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:12:02.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:12:02.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:02.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:12:02.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:02.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:02.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:02.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:12:02.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:12:02.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:02.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:12:02.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=14 2026-02-19T20:12:02.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:12:02.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:12:02.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:12:02.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:02.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:02.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:02.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:02.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:02.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:12:02.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:02.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:12:02.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:12:02.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:12:02.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:12:02.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:12:02.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:12:02.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 14 = '' 2026-02-19T20:12:02.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:12:02.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:12:03.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:03.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:03.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:03.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:03.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:03.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:12:03.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:03.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:12:03.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:12:03.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:12:03.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:12:03.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:12:03.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:12:03.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 14 = '' 2026-02-19T20:12:03.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:12:03.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:12:03.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:03.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:03.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:03.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:03.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:03.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:12:03.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:04.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:12:04.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:12:04.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:12:04.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:12:04.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:12:04.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:12:04.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 14 = '' 2026-02-19T20:12:04.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:12:04.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:12:05.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:05.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:05.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:05.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:05.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:05.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:05.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:12:05.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:12:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=14 2026-02-19T20:12:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 14 = 14 2026-02-19T20:12:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap4 2026-02-19T20:12:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:05.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:05.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap4 2026-02-19T20:12:05.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:12:05.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:12:05.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:12:06.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap4 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 8 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:12:06.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:12:06.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:12:06.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:10:34 2026 2026-02-19T20:12:06.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:12:06.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:12:06.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:12:06.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:12:06.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:12:06.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snap3-clean ']' 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap3-clean 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap3-clean 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:06.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:06.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:12:07.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:12:07.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:07.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:12:07.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:07.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:07.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:07.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap4 2026-02-19T20:12:07.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:07.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:07.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap4 2026-02-19T20:12:07.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:12:07.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:12:07.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:07.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:07.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:12:08.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:12:08.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:08.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:12:08.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:08.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:09.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:09.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap4-clean 2026-02-19T20:12:09.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap4-clean 2026-02-19T20:12:09.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:09.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:09.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:09.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap4-clean 2026-02-19T20:12:09.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:12:09.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8728475173624541) > weight (0.25) 2026-02-19T20:12:09.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4595982538287978) > weight (0.25) 2026-02-19T20:12:09.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9751740266462959) > weight (0.25) 2026-02-19T20:12:09.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.34138675550499464) > weight (0.25) 2026-02-19T20:12:09.336 INFO:tasks.rbd_mirror_thrash:waiting for 95.20830655404993 secs before thrashing 2026-02-19T20:12:09.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:12:09.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:12:09.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:09.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:09.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:09.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:12:09.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:12:09.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:12:09.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:12:09.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 15107072 bytes 2026-02-19T20:12:09.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 24673280 bytes 2026-02-19T20:12:09.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 35155968 bytes 2026-02-19T20:12:09.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 44833792 bytes 2026-02-19T20:12:09.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 61250560 bytes 2026-02-19T20:12:09.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 91535360 bytes 2026-02-19T20:12:09.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 180210688 bytes 2026-02-19T20:12:09.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 200065024 bytes 2026-02-19T20:12:09.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 259210240 bytes 2026-02-19T20:12:10.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 291489792 bytes 2026-02-19T20:12:10.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 318034944 bytes 2026-02-19T20:12:10.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 384201728 bytes 2026-02-19T20:12:10.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 401491968 bytes 2026-02-19T20:12:10.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 410423296 bytes 2026-02-19T20:12:10.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 424290304 bytes 2026-02-19T20:12:10.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 451123200 bytes 2026-02-19T20:12:10.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 489255936 bytes 2026-02-19T20:12:10.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 521756672 bytes 2026-02-19T20:12:10.365 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:12:10.363+0000 7fab1dc3d640 -1 reset not still connected to 0x687c340 2026-02-19T20:12:10.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 575991808 bytes 2026-02-19T20:12:10.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 594354176 bytes 2026-02-19T20:12:10.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 624252928 bytes 2026-02-19T20:12:10.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 651813888 bytes 2026-02-19T20:12:10.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 734986240 bytes 2026-02-19T20:12:10.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 790386688 bytes 2026-02-19T20:12:10.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 829770752 bytes 2026-02-19T20:12:10.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 865830912 bytes 2026-02-19T20:12:10.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 888863744 bytes 2026-02-19T20:12:10.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 923431936 bytes 2026-02-19T20:12:10.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 939579392 bytes 2026-02-19T20:12:10.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 994902016 bytes 2026-02-19T20:12:10.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1014268928 bytes 2026-02-19T20:12:11.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1059649536 bytes 2026-02-19T20:12:11.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1108918272 bytes 2026-02-19T20:12:11.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1161466880 bytes 2026-02-19T20:12:11.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1176771584 bytes 2026-02-19T20:12:11.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1280716800 bytes 2026-02-19T20:12:11.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1363425280 bytes 2026-02-19T20:12:11.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1428917248 bytes 2026-02-19T20:12:11.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1463802880 bytes 2026-02-19T20:12:11.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1520483328 bytes 2026-02-19T20:12:11.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1521097728 bytes 2026-02-19T20:12:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1567176704 bytes 2026-02-19T20:12:11.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1576743936 bytes 2026-02-19T20:12:11.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1634296832 bytes 2026-02-19T20:12:11.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1660033024 bytes 2026-02-19T20:12:11.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1728208896 bytes 2026-02-19T20:12:11.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1822120960 bytes 2026-02-19T20:12:11.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1879776256 bytes 2026-02-19T20:12:11.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1935201280 bytes 2026-02-19T20:12:12.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1973130240 bytes 2026-02-19T20:12:12.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2034155520 bytes 2026-02-19T20:12:12.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2041595904 bytes 2026-02-19T20:12:12.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2090115072 bytes 2026-02-19T20:12:12.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2125767680 bytes 2026-02-19T20:12:12.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2177307648 bytes 2026-02-19T20:12:12.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2245294080 bytes 2026-02-19T20:12:12.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2276723712 bytes 2026-02-19T20:12:12.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2347761664 bytes 2026-02-19T20:12:12.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2378920960 bytes 2026-02-19T20:12:12.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2438413312 bytes 2026-02-19T20:12:12.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2457069568 bytes 2026-02-19T20:12:12.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2464226304 bytes 2026-02-19T20:12:12.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2517439488 bytes 2026-02-19T20:12:12.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2593679360 bytes 2026-02-19T20:12:13.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2619161600 bytes 2026-02-19T20:12:13.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2638915584 bytes 2026-02-19T20:12:13.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2662709248 bytes 2026-02-19T20:12:13.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2698441728 bytes 2026-02-19T20:12:13.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2745644032 bytes 2026-02-19T20:12:13.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2789577728 bytes 2026-02-19T20:12:13.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2833647616 bytes 2026-02-19T20:12:13.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2877927424 bytes 2026-02-19T20:12:13.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2886121472 bytes 2026-02-19T20:12:13.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2915547136 bytes 2026-02-19T20:12:13.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3007646720 bytes 2026-02-19T20:12:13.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3059010560 bytes 2026-02-19T20:12:13.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3121192960 bytes 2026-02-19T20:12:13.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3175408640 bytes 2026-02-19T20:12:13.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3259232256 bytes 2026-02-19T20:12:13.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3288532992 bytes 2026-02-19T20:12:13.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3313723392 bytes 2026-02-19T20:12:13.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3379064832 bytes 2026-02-19T20:12:13.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3440884736 bytes 2026-02-19T20:12:14.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3506776064 bytes 2026-02-19T20:12:14.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3530927104 bytes 2026-02-19T20:12:14.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3572893696 bytes 2026-02-19T20:12:14.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3582345216 bytes 2026-02-19T20:12:14.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3637126144 bytes 2026-02-19T20:12:14.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3649294336 bytes 2026-02-19T20:12:14.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3751704576 bytes 2026-02-19T20:12:14.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3803352064 bytes 2026-02-19T20:12:14.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3875415040 bytes 2026-02-19T20:12:14.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3898528768 bytes 2026-02-19T20:12:14.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3916771328 bytes 2026-02-19T20:12:14.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3937606656 bytes 2026-02-19T20:12:14.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3970582528 bytes 2026-02-19T20:12:14.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3971180544 bytes 2026-02-19T20:12:14.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4008952832 bytes 2026-02-19T20:12:14.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4035249152 bytes 2026-02-19T20:12:14.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4058445824 bytes 2026-02-19T20:12:14.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4128861184 bytes 2026-02-19T20:12:14.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4223766528 bytes 2026-02-19T20:12:14.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4271372288 bytes 2026-02-19T20:12:15.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4366895104 bytes 2026-02-19T20:12:15.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4392159232 bytes 2026-02-19T20:12:15.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4456086528 bytes 2026-02-19T20:12:15.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4479302656 bytes 2026-02-19T20:12:15.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4505559040 bytes 2026-02-19T20:12:15.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4558477312 bytes 2026-02-19T20:12:15.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4600492032 bytes 2026-02-19T20:12:15.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4601111552 bytes 2026-02-19T20:12:15.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4665461760 bytes 2026-02-19T20:12:15.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4683098112 bytes 2026-02-19T20:12:15.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4705985536 bytes 2026-02-19T20:12:15.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4754488320 bytes 2026-02-19T20:12:15.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4787303424 bytes 2026-02-19T20:12:15.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4826203136 bytes 2026-02-19T20:12:15.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4852926464 bytes 2026-02-19T20:12:15.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4877830144 bytes 2026-02-19T20:12:15.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4930834432 bytes 2026-02-19T20:12:15.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4978513920 bytes 2026-02-19T20:12:15.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5022280704 bytes 2026-02-19T20:12:15.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5083534336 bytes 2026-02-19T20:12:16.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5169321984 bytes 2026-02-19T20:12:16.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5218300928 bytes 2026-02-19T20:12:16.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5270196224 bytes 2026-02-19T20:12:16.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5291581440 bytes 2026-02-19T20:12:16.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5349998592 bytes 2026-02-19T20:12:16.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5376151552 bytes 2026-02-19T20:12:16.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5385146368 bytes 2026-02-19T20:12:16.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5442195456 bytes 2026-02-19T20:12:16.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5515017216 bytes 2026-02-19T20:12:16.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5549556736 bytes 2026-02-19T20:12:16.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5580845056 bytes 2026-02-19T20:12:16.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5623336960 bytes 2026-02-19T20:12:16.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5647291392 bytes 2026-02-19T20:12:16.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5673910272 bytes 2026-02-19T20:12:16.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5712788480 bytes 2026-02-19T20:12:16.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5754510336 bytes 2026-02-19T20:12:16.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5796574208 bytes 2026-02-19T20:12:16.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5891120128 bytes 2026-02-19T20:12:16.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5959169024 bytes 2026-02-19T20:12:17.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5994730496 bytes 2026-02-19T20:12:17.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6071242752 bytes 2026-02-19T20:12:17.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6097617920 bytes 2026-02-19T20:12:17.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6164468736 bytes 2026-02-19T20:12:17.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6206942208 bytes 2026-02-19T20:12:17.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6232386560 bytes 2026-02-19T20:12:17.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6264647680 bytes 2026-02-19T20:12:17.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6371289088 bytes 2026-02-19T20:12:17.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6500813824 bytes 2026-02-19T20:12:17.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6501448704 bytes 2026-02-19T20:12:17.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6534218752 bytes 2026-02-19T20:12:17.638 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:12:17.636+0000 7fab1dc3d640 -1 reset not still connected to 0x687c340 2026-02-19T20:12:17.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6581907456 bytes 2026-02-19T20:12:17.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6657383424 bytes 2026-02-19T20:12:17.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6704649216 bytes 2026-02-19T20:12:17.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6737289216 bytes 2026-02-19T20:12:17.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6768898048 bytes 2026-02-19T20:12:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6792104960 bytes 2026-02-19T20:12:17.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6861409280 bytes 2026-02-19T20:12:18.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6896950272 bytes 2026-02-19T20:12:18.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6907447296 bytes 2026-02-19T20:12:18.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6967090176 bytes 2026-02-19T20:12:18.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6994458624 bytes 2026-02-19T20:12:18.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7036991488 bytes 2026-02-19T20:12:18.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7043755008 bytes 2026-02-19T20:12:18.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7044359168 bytes 2026-02-19T20:12:18.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7101665280 bytes 2026-02-19T20:12:18.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7130083328 bytes 2026-02-19T20:12:18.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7218409472 bytes 2026-02-19T20:12:18.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7292876800 bytes 2026-02-19T20:12:18.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7350412288 bytes 2026-02-19T20:12:18.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7381515264 bytes 2026-02-19T20:12:18.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7382131712 bytes 2026-02-19T20:12:18.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7458945024 bytes 2026-02-19T20:12:18.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7529732096 bytes 2026-02-19T20:12:18.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7543703552 bytes 2026-02-19T20:12:18.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7625763840 bytes 2026-02-19T20:12:18.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7667387392 bytes 2026-02-19T20:12:19.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7746142208 bytes 2026-02-19T20:12:19.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7808275456 bytes 2026-02-19T20:12:19.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7857263616 bytes 2026-02-19T20:12:19.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7890709504 bytes 2026-02-19T20:12:19.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7908257792 bytes 2026-02-19T20:12:19.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7943060480 bytes 2026-02-19T20:12:19.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7943667712 bytes 2026-02-19T20:12:19.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8037975040 bytes 2026-02-19T20:12:19.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8057880576 bytes 2026-02-19T20:12:19.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8068399104 bytes 2026-02-19T20:12:19.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8076614656 bytes 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:12:19.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:12:19.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:12:19.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:12:19.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:12:20.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:12:20.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:20.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:20.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:20.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:12:20.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:12:20.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:12:20.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:12:20.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:12:20.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:12:20.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:12:20.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:12:20.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:12:20.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":17886958.93,"bytes_per_snapshot":536292352.0,"last_snapshot_bytes":536608768,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531875,"remote_snapshot_timestamp":1771531875,"replay_state":"idle"} 2026-02-19T20:12:20.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:12:20.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:11:56 2026-02-19T20:12:20.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:12:20.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:12:20.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+stopped 2026-02-19T20:12:20.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: local image is primary 2026-02-19T20:12:20.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:11:56' 2026-02-19T20:12:20.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:12:20.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:12:20.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:12:20.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":17886958.93,"bytes_per_snapshot":536292352.0,"last_snapshot_bytes":536608768,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531875,"remote_snapshot_timestamp":1771531875,"replay_state":"idle"} 2026-02-19T20:12:20.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:12:20.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:12:20.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:12:20.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:12:20.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:12:20.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:12:20.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:12:20.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:12:20.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:11:56' 2026-02-19T20:12:20.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:11:56' 2026-02-19T20:12:20.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:20.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:20.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:20.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:12:20.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:11:56' 2026-02-19T20:12:20.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:12:20.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:12:20.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:12:20.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='OKOKOK142024288mirror.0trial08020.3.0-5336-g802bf111falseOK42054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK44074410mirror.3trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":17886958.93,"bytes_per_snapshot":536292352.0,"last_snapshot_bytes":536608768,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531875,"remote_snapshot_timestamp":1771531875,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:11:56cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3up+stoppedlocal image is primary2026-02-19 20:11:56' 2026-02-19T20:12:20.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:12:20.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:12:20.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:11:56' 2026-02-19T20:12:20.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:12:20.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:12:20.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:12:20.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:12:20.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:12:20.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:20.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:20.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap5 2026-02-19T20:12:20.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap5 2026-02-19T20:12:20.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:20.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:20.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:20.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap5 2026-02-19T20:12:20.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:12:49.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:12:49.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:12:49.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:12:50.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:12:50.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:12:50.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:12:50.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:12:50.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:12:50.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:12:50.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:12:50.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:12:50.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:12:50.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:12:50.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:12:50.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:12:50.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:12:50.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:12:50.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:12:50.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:12:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:12:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:12:50.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:12:50.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:12:50.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:12:50.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:12:50.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:12:50.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:12:50.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:12:50.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:12:50.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:12:50.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:12:50.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:12:50.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:12:50.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:12:50.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:12:50.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:12:50.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:12:50.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:12:50.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:12:50.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:12:50.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:12:50.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:50.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:12:50.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:12:50.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:12:50.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:50.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:50.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:50.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:12:52.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 17 2026-02-19T20:12:52.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:12:52.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:12:52.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:52.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:12:52.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:12:52.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:52.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:12:52.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:12:52.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:12:52.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:12:52.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:12:52.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:12:52.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:12:52.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:52.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:12:52.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:52.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:52.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:52.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:12:52.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:12:52.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:52.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:12:52.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=17 2026-02-19T20:12:52.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:12:52.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:12:52.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:12:52.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:52.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:52.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:52.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:52.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:52.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:12:52.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:53.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:12:53.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:12:53.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:12:53.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:12:53.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:12:53.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:12:53.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 17 = '' 2026-02-19T20:12:53.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:12:53.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:12:53.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:53.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:12:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:53.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:12:53.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:12:53.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:12:53.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:12:53.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:12:53.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:12:53.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 17 = '' 2026-02-19T20:12:53.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:12:53.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:12:54.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:54.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:54.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:54.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:54.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:54.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:12:54.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:54.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:12:54.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:12:54.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:12:54.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:12:54.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:12:54.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:12:54.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 17 = '' 2026-02-19T20:12:54.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:12:54.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:12:55.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:55.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:55.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:55.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:55.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:12:55.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:12:55.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:12:55.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:12:55.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=17 2026-02-19T20:12:55.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 17 = 17 2026-02-19T20:12:55.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:55.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap5 2026-02-19T20:12:55.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:12:55.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:55.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:55.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap5 2026-02-19T20:12:55.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:12:55.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:12:55.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:12:56.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap5 2026-02-19T20:12:56.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:12:56.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:12:56.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:12:56.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 10 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:12:07 2026 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:12:56.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:12:56.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:12:56.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:12:56.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:12:56.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:12:56.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:56.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snap4-clean ']' 2026-02-19T20:12:56.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap4-clean 2026-02-19T20:12:56.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:56.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:56.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap4-clean 2026-02-19T20:12:56.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:12:56.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:12:56.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:56.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:56.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:12:57.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:12:57.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:57.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:12:57.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:58.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:58.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:58.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap5 2026-02-19T20:12:58.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:58.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:58.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap5 2026-02-19T20:12:58.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:12:58.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:12:58.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:58.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:58.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:12:58.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:12:58.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:58.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:12:58.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:59.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:12:59.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:12:59.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap5-clean 2026-02-19T20:12:59.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap5-clean 2026-02-19T20:12:59.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:59.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:59.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:59.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap5-clean 2026-02-19T20:12:59.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:12:59.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:12:59.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:12:59.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:12:59.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:12:59.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:12:59.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:12:59.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:12:59.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:12:59.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:12:59.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 63724544 bytes 2026-02-19T20:12:59.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 97753088 bytes 2026-02-19T20:12:59.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 147371008 bytes 2026-02-19T20:12:59.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 204169216 bytes 2026-02-19T20:12:59.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 225504256 bytes 2026-02-19T20:12:59.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 282180608 bytes 2026-02-19T20:13:00.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 335058944 bytes 2026-02-19T20:13:00.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 369895424 bytes 2026-02-19T20:13:00.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 411341824 bytes 2026-02-19T20:13:00.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 422439936 bytes 2026-02-19T20:13:00.230 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 491271168 bytes 2026-02-19T20:13:00.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 494389248 bytes 2026-02-19T20:13:00.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 562161664 bytes 2026-02-19T20:13:00.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 597010432 bytes 2026-02-19T20:13:00.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 708727808 bytes 2026-02-19T20:13:00.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 816142336 bytes 2026-02-19T20:13:00.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 853023744 bytes 2026-02-19T20:13:00.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 934184960 bytes 2026-02-19T20:13:00.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 937170944 bytes 2026-02-19T20:13:00.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 951404544 bytes 2026-02-19T20:13:00.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 990246912 bytes 2026-02-19T20:13:00.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1029452800 bytes 2026-02-19T20:13:00.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1054523392 bytes 2026-02-19T20:13:00.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1099574272 bytes 2026-02-19T20:13:00.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1103873024 bytes 2026-02-19T20:13:01.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1153628160 bytes 2026-02-19T20:13:01.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1203435520 bytes 2026-02-19T20:13:01.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1280330752 bytes 2026-02-19T20:13:01.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1356787712 bytes 2026-02-19T20:13:01.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1421237248 bytes 2026-02-19T20:13:01.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1450582016 bytes 2026-02-19T20:13:01.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1542523904 bytes 2026-02-19T20:13:01.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1597084672 bytes 2026-02-19T20:13:01.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1609755648 bytes 2026-02-19T20:13:01.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1655546880 bytes 2026-02-19T20:13:01.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1694783488 bytes 2026-02-19T20:13:01.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1767217152 bytes 2026-02-19T20:13:01.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1844114432 bytes 2026-02-19T20:13:01.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1877856256 bytes 2026-02-19T20:13:01.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1932647424 bytes 2026-02-19T20:13:01.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1957156864 bytes 2026-02-19T20:13:01.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1981790208 bytes 2026-02-19T20:13:02.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2008208384 bytes 2026-02-19T20:13:02.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2050412544 bytes 2026-02-19T20:13:02.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2058663936 bytes 2026-02-19T20:13:02.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2086788096 bytes 2026-02-19T20:13:02.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2128750592 bytes 2026-02-19T20:13:02.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2172808192 bytes 2026-02-19T20:13:02.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2223134720 bytes 2026-02-19T20:13:02.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2277904384 bytes 2026-02-19T20:13:02.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2347158528 bytes 2026-02-19T20:13:02.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2378146816 bytes 2026-02-19T20:13:02.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2384156672 bytes 2026-02-19T20:13:02.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2443990016 bytes 2026-02-19T20:13:02.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2483877888 bytes 2026-02-19T20:13:02.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2535701504 bytes 2026-02-19T20:13:02.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2599533568 bytes 2026-02-19T20:13:02.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2623431680 bytes 2026-02-19T20:13:02.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2664714240 bytes 2026-02-19T20:13:02.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2684258304 bytes 2026-02-19T20:13:02.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2707759104 bytes 2026-02-19T20:13:02.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2738816000 bytes 2026-02-19T20:13:03.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2739396608 bytes 2026-02-19T20:13:03.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2758016000 bytes 2026-02-19T20:13:03.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2791028736 bytes 2026-02-19T20:13:03.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2853900288 bytes 2026-02-19T20:13:03.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2919563264 bytes 2026-02-19T20:13:03.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2920167424 bytes 2026-02-19T20:13:03.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2958330880 bytes 2026-02-19T20:13:03.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3012538368 bytes 2026-02-19T20:13:03.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3046966272 bytes 2026-02-19T20:13:03.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3116065792 bytes 2026-02-19T20:13:03.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3153500160 bytes 2026-02-19T20:13:03.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3201824768 bytes 2026-02-19T20:13:03.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3311155200 bytes 2026-02-19T20:13:03.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3328430080 bytes 2026-02-19T20:13:03.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3390074880 bytes 2026-02-19T20:13:03.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3399606272 bytes 2026-02-19T20:13:03.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3466455040 bytes 2026-02-19T20:13:03.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3520253952 bytes 2026-02-19T20:13:04.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3593726976 bytes 2026-02-19T20:13:04.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3663442944 bytes 2026-02-19T20:13:04.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3705617408 bytes 2026-02-19T20:13:04.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3723001856 bytes 2026-02-19T20:13:04.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3766136832 bytes 2026-02-19T20:13:04.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3780182016 bytes 2026-02-19T20:13:04.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3802780672 bytes 2026-02-19T20:13:04.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3809901568 bytes 2026-02-19T20:13:04.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3817268224 bytes 2026-02-19T20:13:04.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3912144896 bytes 2026-02-19T20:13:04.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4002996224 bytes 2026-02-19T20:13:04.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4024045568 bytes 2026-02-19T20:13:04.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4089158656 bytes 2026-02-19T20:13:04.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4165947392 bytes 2026-02-19T20:13:04.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4266251264 bytes 2026-02-19T20:13:04.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4336436224 bytes 2026-02-19T20:13:04.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4356173824 bytes 2026-02-19T20:13:04.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4421389312 bytes 2026-02-19T20:13:05.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4552046592 bytes 2026-02-19T20:13:05.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4566581248 bytes 2026-02-19T20:13:05.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4634973184 bytes 2026-02-19T20:13:05.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4674793472 bytes 2026-02-19T20:13:05.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4737927168 bytes 2026-02-19T20:13:05.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4769228800 bytes 2026-02-19T20:13:05.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4769831936 bytes 2026-02-19T20:13:05.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4809734144 bytes 2026-02-19T20:13:05.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4837637120 bytes 2026-02-19T20:13:05.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4870725632 bytes 2026-02-19T20:13:05.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4893836288 bytes 2026-02-19T20:13:05.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4945253376 bytes 2026-02-19T20:13:05.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5018297344 bytes 2026-02-19T20:13:05.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5027616768 bytes 2026-02-19T20:13:05.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5122081792 bytes 2026-02-19T20:13:05.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5257750528 bytes 2026-02-19T20:13:05.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5322778624 bytes 2026-02-19T20:13:06.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5324797952 bytes 2026-02-19T20:13:06.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5329270784 bytes 2026-02-19T20:13:06.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5381116928 bytes 2026-02-19T20:13:06.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5412005888 bytes 2026-02-19T20:13:06.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5446341632 bytes 2026-02-19T20:13:06.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5503713280 bytes 2026-02-19T20:13:06.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5554167808 bytes 2026-02-19T20:13:06.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5591360512 bytes 2026-02-19T20:13:06.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5601499136 bytes 2026-02-19T20:13:06.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5651405824 bytes 2026-02-19T20:13:06.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5710423040 bytes 2026-02-19T20:13:06.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5792809984 bytes 2026-02-19T20:13:06.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5805541376 bytes 2026-02-19T20:13:06.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5894953984 bytes 2026-02-19T20:13:06.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5941155840 bytes 2026-02-19T20:13:06.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 6013078528 bytes 2026-02-19T20:13:06.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 6085346304 bytes 2026-02-19T20:13:07.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 6149432320 bytes 2026-02-19T20:13:07.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6152000512 bytes 2026-02-19T20:13:07.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6202887168 bytes 2026-02-19T20:13:07.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6257412096 bytes 2026-02-19T20:13:07.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6267842560 bytes 2026-02-19T20:13:07.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6356428800 bytes 2026-02-19T20:13:07.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6394968064 bytes 2026-02-19T20:13:07.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6423569408 bytes 2026-02-19T20:13:07.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6461119488 bytes 2026-02-19T20:13:07.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6581240832 bytes 2026-02-19T20:13:07.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6625547264 bytes 2026-02-19T20:13:07.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6630435840 bytes 2026-02-19T20:13:07.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6688432128 bytes 2026-02-19T20:13:07.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6753939456 bytes 2026-02-19T20:13:07.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6775485440 bytes 2026-02-19T20:13:07.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6799797248 bytes 2026-02-19T20:13:07.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6846362624 bytes 2026-02-19T20:13:07.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6908242944 bytes 2026-02-19T20:13:07.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6997022720 bytes 2026-02-19T20:13:08.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 7009809408 bytes 2026-02-19T20:13:08.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 7028352000 bytes 2026-02-19T20:13:08.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 7064741888 bytes 2026-02-19T20:13:08.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 7096194048 bytes 2026-02-19T20:13:08.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7115405312 bytes 2026-02-19T20:13:08.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7146928128 bytes 2026-02-19T20:13:08.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7175264256 bytes 2026-02-19T20:13:08.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7232021504 bytes 2026-02-19T20:13:08.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7236954112 bytes 2026-02-19T20:13:08.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7303914496 bytes 2026-02-19T20:13:08.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7360712704 bytes 2026-02-19T20:13:08.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7446372352 bytes 2026-02-19T20:13:08.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7490036736 bytes 2026-02-19T20:13:08.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7540519936 bytes 2026-02-19T20:13:08.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7556970496 bytes 2026-02-19T20:13:08.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7601446912 bytes 2026-02-19T20:13:08.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7669888000 bytes 2026-02-19T20:13:08.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7672086528 bytes 2026-02-19T20:13:08.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7725310976 bytes 2026-02-19T20:13:09.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7759552512 bytes 2026-02-19T20:13:09.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7781258240 bytes 2026-02-19T20:13:09.077 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7793946624 bytes 2026-02-19T20:13:09.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7841271808 bytes 2026-02-19T20:13:09.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7869107200 bytes 2026-02-19T20:13:09.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7939857408 bytes 2026-02-19T20:13:09.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 8027730944 bytes 2026-02-19T20:13:09.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 8138946560 bytes 2026-02-19T20:13:09.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 8159572992 bytes 2026-02-19T20:13:09.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 8181288960 bytes 2026-02-19T20:13:09.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 8220626944 bytes 2026-02-19T20:13:09.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 8293783552 bytes 2026-02-19T20:13:09.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 8365444096 bytes 2026-02-19T20:13:09.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 8395314176 bytes 2026-02-19T20:13:09.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 8440495104 bytes 2026-02-19T20:13:09.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8455179264 bytes 2026-02-19T20:13:09.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8533028864 bytes 2026-02-19T20:13:09.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8577987584 bytes 2026-02-19T20:13:10.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8624068608 bytes 2026-02-19T20:13:10.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8672741376 bytes 2026-02-19T20:13:10.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8719925248 bytes 2026-02-19T20:13:10.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8778203136 bytes 2026-02-19T20:13:10.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8785348608 bytes 2026-02-19T20:13:10.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8834291712 bytes 2026-02-19T20:13:10.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8869330944 bytes 2026-02-19T20:13:10.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8915948544 bytes 2026-02-19T20:13:10.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8931587072 bytes 2026-02-19T20:13:10.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8972061696 bytes 2026-02-19T20:13:10.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 9000569856 bytes 2026-02-19T20:13:10.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 9001205760 bytes 2026-02-19T20:13:10.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 9022072832 bytes 2026-02-19T20:13:10.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 9072126976 bytes 2026-02-19T20:13:10.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 9109555200 bytes 2026-02-19T20:13:10.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 9187440640 bytes 2026-02-19T20:13:10.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 9218414592 bytes 2026-02-19T20:13:10.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 9396396032 bytes 2026-02-19T20:13:11.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 9461120000 bytes 2026-02-19T20:13:11.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 9551411200 bytes 2026-02-19T20:13:11.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 9589152768 bytes 2026-02-19T20:13:11.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 9642099712 bytes 2026-02-19T20:13:11.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 9698207744 bytes 2026-02-19T20:13:11.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 9717438464 bytes 2026-02-19T20:13:11.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 9776756736 bytes 2026-02-19T20:13:11.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 9787316224 bytes 2026-02-19T20:13:11.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 9796592640 bytes 2026-02-19T20:13:11.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 9834532864 bytes 2026-02-19T20:13:11.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 9841259520 bytes 2026-02-19T20:13:11.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 9865239552 bytes 2026-02-19T20:13:11.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 9958152192 bytes 2026-02-19T20:13:11.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 10012933120 bytes 2026-02-19T20:13:11.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 10034353152 bytes 2026-02-19T20:13:11.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 10072989696 bytes 2026-02-19T20:13:11.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 10145364992 bytes 2026-02-19T20:13:11.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 10192022528 bytes 2026-02-19T20:13:11.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 10198976512 bytes 2026-02-19T20:13:11.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22600 IOs, 10218717184 bytes 2026-02-19T20:13:12.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22700 IOs, 10242797568 bytes 2026-02-19T20:13:12.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22800 IOs, 10302042112 bytes 2026-02-19T20:13:12.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22900 IOs, 10317607936 bytes 2026-02-19T20:13:12.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23000 IOs, 10397617152 bytes 2026-02-19T20:13:12.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23100 IOs, 10410754048 bytes 2026-02-19T20:13:12.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23200 IOs, 10469338112 bytes 2026-02-19T20:13:12.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23300 IOs, 10565278720 bytes 2026-02-19T20:13:12.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23400 IOs, 10647053312 bytes 2026-02-19T20:13:12.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23500 IOs, 10686787584 bytes 2026-02-19T20:13:12.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:13:12.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:13:12.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:13:12.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:13:12.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:13:12.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:12.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:12.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:12.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:13:12.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:13:12.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:13:12.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:13:12.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:13:13.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:13:13.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:13.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:13.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:13.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:13:13.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:13:13.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:13:13.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:13:13.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:13:13.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:13:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:13:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:13:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:13:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":17858730.67,"bytes_per_snapshot":533826560.0,"last_snapshot_bytes":535761920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531971,"remote_snapshot_timestamp":1771531971,"replay_state":"idle"} 2026-02-19T20:13:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:13:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:12:56 2026-02-19T20:13:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:13:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:13:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+stopped 2026-02-19T20:13:13.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: local image is primary 2026-02-19T20:13:13.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:12:56' 2026-02-19T20:13:13.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:13:13.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:13:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:13:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":17858730.67,"bytes_per_snapshot":533826560.0,"last_snapshot_bytes":535761920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531971,"remote_snapshot_timestamp":1771531971,"replay_state":"idle"} 2026-02-19T20:13:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:13:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:13:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:13:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:13:13.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:13:13.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:13:13.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:13:13.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:13:13.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:12:56' 2026-02-19T20:13:13.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:12:56' 2026-02-19T20:13:13.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:13.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:13.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:13.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:13:13.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:12:56' 2026-02-19T20:13:13.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:13:13.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:13:13.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:13:13.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='OKOKOK142024288mirror.0trial08020.3.0-5336-g802bf111falseOK42054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK44074410mirror.3trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":17858730.67,"bytes_per_snapshot":533826560.0,"last_snapshot_bytes":535761920,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771531971,"remote_snapshot_timestamp":1771531971,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:12:56cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3up+stoppedlocal image is primary2026-02-19 20:12:56' 2026-02-19T20:13:13.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:13:13.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:13:13.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:12:56' 2026-02-19T20:13:13.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:13:13.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:13:13.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:13:13.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:13:13.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:13:13.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:13:13.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:13:13.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap6 2026-02-19T20:13:13.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap6 2026-02-19T20:13:13.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:13:13.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:13.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:13.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap6 2026-02-19T20:13:13.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:13:21.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8857408287203827) > weight (0.25) 2026-02-19T20:13:21.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46189984749861) > weight (0.25) 2026-02-19T20:13:21.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3688548288794432) > weight (0.25) 2026-02-19T20:13:21.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7229568466345079) > weight (0.25) 2026-02-19T20:13:21.048 INFO:tasks.rbd_mirror_thrash:waiting for 73.25437478214705 secs before thrashing 2026-02-19T20:13:43.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:13:43.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:43.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:43.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:43.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:13:43.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:43.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:43.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:43.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:13:43.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:13:43.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:13:43.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:13:44.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:13:44.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:13:44.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:13:44.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:13:44.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:13:44.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:13:44.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:13:44.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:13:44.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:13:44.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:13:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:13:44.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:13:44.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:13:44.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:13:44.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:13:44.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:13:44.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:13:44.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:13:44.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:13:44.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:13:44.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:13:44.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:13:44.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:13:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:13:44.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:13:44.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:13:44.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:13:44.550 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2026-02-19T20:13:44.550 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2026-02-19T20:13:44.550 INFO:tasks.rbd_mirror_thrash:waiting for 7.000393241312489 secs before reviving daemons 2026-02-19T20:13:44.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:13:44.549+0000 7fe9d3336640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50161) UID: 1000 2026-02-19T20:13:44.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:13:44.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:13:44.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:13:44.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:13:44.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:13:44.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:13:44.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:13:44.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:13:44.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:13:44.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:13:44.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:13:44.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:13:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:13:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:13:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:13:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:13:44.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:13:44.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:13:44.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:13:44.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:13:44.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:13:44.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:13:44.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:13:44.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:13:44.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:13:44.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:44.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:13:46.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 20 2026-02-19T20:13:46.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:13:46.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:13:46.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:13:46.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:13:46.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:13:46.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:46.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:13:46.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:13:46.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:13:46.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:13:46.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:13:46.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:13:46.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:13:46.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:13:46.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:13:46.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:13:46.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:46.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:46.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:13:46.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:13:46.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:13:46.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:13:46.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=20 2026-02-19T20:13:46.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:13:46.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:13:46.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:13:46.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:13:46.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:46.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:46.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:46.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:13:46.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:13:46.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:13:46.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:13:46.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:13:46.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:13:46.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:13:46.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:13:46.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:13:46.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 20 = '' 2026-02-19T20:13:46.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:13:46.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:13:47.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:13:47.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:47.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:47.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:47.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:13:47.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:13:47.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:13:47.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:13:47.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:13:47.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:13:47.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:13:47.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:13:47.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:13:47.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 20 = '' 2026-02-19T20:13:47.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:13:47.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:13:48.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:13:48.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:48.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:48.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:48.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:13:48.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:13:48.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:13:48.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:13:48.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:13:48.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:13:48.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:13:48.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:13:48.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:13:48.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 20 = '' 2026-02-19T20:13:48.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:13:48.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:13:49.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:13:49.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:49.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:49.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:49.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:13:49.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:13:49.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:13:49.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:13:49.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=20 2026-02-19T20:13:49.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 20 = 20 2026-02-19T20:13:49.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:13:49.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap6 2026-02-19T20:13:49.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:13:49.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:49.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:49.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap6 2026-02-19T20:13:49.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:13:49.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:13:49.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:13:50.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap6 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 12 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:12:07 2026 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:13:50.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snap5-clean ']' 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap5-clean 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap5-clean 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:50.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:50.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:13:51.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:13:51.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:51.551 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2026-02-19T20:13:51.552 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:13:51.552 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2026-02-19T20:13:51.552 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2026-02-19T20:13:51.552 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2026-02-19T20:13:51.555 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2026-02-19T20:13:51.555 INFO:tasks.rbd_mirror_thrash:waiting for 69.28443304687107 secs before thrashing 2026-02-19T20:13:51.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:13:51.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:51.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:51.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:13:51.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap6 2026-02-19T20:13:51.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:51.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:51.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap6 2026-02-19T20:13:51.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:13:51.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:13:51.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:51.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:51.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:13:52.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:13:52.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:52.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:13:52.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:52.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:13:53.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:13:53.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap6-clean 2026-02-19T20:13:53.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap6-clean 2026-02-19T20:13:53.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:13:53.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:53.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:53.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap6-clean 2026-02-19T20:13:53.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:13:53.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:13:53.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:13:53.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:13:53.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:13:53.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:13:53.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:13:53.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:13:53.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:13:53.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:13:53.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 70670336 bytes 2026-02-19T20:13:53.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 109856768 bytes 2026-02-19T20:13:53.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 174226432 bytes 2026-02-19T20:13:53.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 274125824 bytes 2026-02-19T20:13:53.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 325240832 bytes 2026-02-19T20:13:54.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 351124480 bytes 2026-02-19T20:13:54.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 354040832 bytes 2026-02-19T20:13:54.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 379854848 bytes 2026-02-19T20:13:54.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 453326848 bytes 2026-02-19T20:13:54.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 472005632 bytes 2026-02-19T20:13:54.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 514647040 bytes 2026-02-19T20:13:54.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 579777536 bytes 2026-02-19T20:13:54.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 608887808 bytes 2026-02-19T20:13:54.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 647826432 bytes 2026-02-19T20:13:54.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 658081792 bytes 2026-02-19T20:13:54.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 683307008 bytes 2026-02-19T20:13:54.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 683905024 bytes 2026-02-19T20:13:54.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 757437440 bytes 2026-02-19T20:13:54.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 831137792 bytes 2026-02-19T20:13:54.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 845858816 bytes 2026-02-19T20:13:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 868586496 bytes 2026-02-19T20:13:54.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 902035456 bytes 2026-02-19T20:13:54.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 929035264 bytes 2026-02-19T20:13:54.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 972366848 bytes 2026-02-19T20:13:54.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1026975744 bytes 2026-02-19T20:13:54.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1048707072 bytes 2026-02-19T20:13:55.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1105441792 bytes 2026-02-19T20:13:55.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1143920640 bytes 2026-02-19T20:13:55.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1214716928 bytes 2026-02-19T20:13:55.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1249797120 bytes 2026-02-19T20:13:55.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1305796608 bytes 2026-02-19T20:13:55.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1341532160 bytes 2026-02-19T20:13:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1369794560 bytes 2026-02-19T20:13:55.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1401641984 bytes 2026-02-19T20:13:55.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1440010240 bytes 2026-02-19T20:13:55.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1479001088 bytes 2026-02-19T20:13:55.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1547636736 bytes 2026-02-19T20:13:55.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1552463872 bytes 2026-02-19T20:13:55.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1570272256 bytes 2026-02-19T20:13:55.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1583889408 bytes 2026-02-19T20:13:55.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1661774848 bytes 2026-02-19T20:13:55.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1692887040 bytes 2026-02-19T20:13:55.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1720253440 bytes 2026-02-19T20:13:55.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1731814400 bytes 2026-02-19T20:13:55.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1759801344 bytes 2026-02-19T20:13:56.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1797225472 bytes 2026-02-19T20:13:56.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1846470656 bytes 2026-02-19T20:13:56.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1863196672 bytes 2026-02-19T20:13:56.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1872108544 bytes 2026-02-19T20:13:56.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1953740800 bytes 2026-02-19T20:13:56.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2004156416 bytes 2026-02-19T20:13:56.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2026760192 bytes 2026-02-19T20:13:56.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2072773632 bytes 2026-02-19T20:13:56.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2097070080 bytes 2026-02-19T20:13:56.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2149597184 bytes 2026-02-19T20:13:56.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2168206336 bytes 2026-02-19T20:13:56.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2191360000 bytes 2026-02-19T20:13:56.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2211148800 bytes 2026-02-19T20:13:56.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2280437760 bytes 2026-02-19T20:13:56.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2338552832 bytes 2026-02-19T20:13:56.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2373353472 bytes 2026-02-19T20:13:56.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2413786112 bytes 2026-02-19T20:13:56.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2439060480 bytes 2026-02-19T20:13:56.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2439660544 bytes 2026-02-19T20:13:56.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2464217088 bytes 2026-02-19T20:13:57.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2490683392 bytes 2026-02-19T20:13:57.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2530313216 bytes 2026-02-19T20:13:57.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2599589888 bytes 2026-02-19T20:13:57.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2647255040 bytes 2026-02-19T20:13:57.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2708015104 bytes 2026-02-19T20:13:57.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2722313216 bytes 2026-02-19T20:13:57.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2760760320 bytes 2026-02-19T20:13:57.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2832875520 bytes 2026-02-19T20:13:57.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2833483776 bytes 2026-02-19T20:13:57.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2895350784 bytes 2026-02-19T20:13:57.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2998583296 bytes 2026-02-19T20:13:57.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3007095808 bytes 2026-02-19T20:13:57.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3069853696 bytes 2026-02-19T20:13:57.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3104806912 bytes 2026-02-19T20:13:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3105420288 bytes 2026-02-19T20:13:57.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3128824832 bytes 2026-02-19T20:13:57.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3160253440 bytes 2026-02-19T20:13:57.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3176936448 bytes 2026-02-19T20:13:57.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3237501952 bytes 2026-02-19T20:13:58.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3311797248 bytes 2026-02-19T20:13:58.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3362098176 bytes 2026-02-19T20:13:58.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3449244672 bytes 2026-02-19T20:13:58.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3449841664 bytes 2026-02-19T20:13:58.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3463386112 bytes 2026-02-19T20:13:58.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3562011648 bytes 2026-02-19T20:13:58.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3623177216 bytes 2026-02-19T20:13:58.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3687772160 bytes 2026-02-19T20:13:58.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3740680192 bytes 2026-02-19T20:13:58.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3803547648 bytes 2026-02-19T20:13:58.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3845595136 bytes 2026-02-19T20:13:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3913732096 bytes 2026-02-19T20:13:58.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3969281024 bytes 2026-02-19T20:13:58.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3987777536 bytes 2026-02-19T20:13:58.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4028773376 bytes 2026-02-19T20:13:58.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4039947264 bytes 2026-02-19T20:13:58.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4058659840 bytes 2026-02-19T20:13:59.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4118434816 bytes 2026-02-19T20:13:59.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4153478144 bytes 2026-02-19T20:13:59.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4176814080 bytes 2026-02-19T20:13:59.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4237053952 bytes 2026-02-19T20:13:59.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4269107200 bytes 2026-02-19T20:13:59.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4331667456 bytes 2026-02-19T20:13:59.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4382645248 bytes 2026-02-19T20:13:59.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4400921600 bytes 2026-02-19T20:13:59.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4434794496 bytes 2026-02-19T20:13:59.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4488826880 bytes 2026-02-19T20:13:59.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4552308736 bytes 2026-02-19T20:13:59.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4576488448 bytes 2026-02-19T20:13:59.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4603448320 bytes 2026-02-19T20:13:59.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4639138816 bytes 2026-02-19T20:13:59.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4678246400 bytes 2026-02-19T20:13:59.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4683663360 bytes 2026-02-19T20:13:59.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4701022208 bytes 2026-02-19T20:13:59.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4748112896 bytes 2026-02-19T20:13:59.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4749026304 bytes 2026-02-19T20:13:59.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4784410624 bytes 2026-02-19T20:14:00.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4831626240 bytes 2026-02-19T20:14:00.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4856406016 bytes 2026-02-19T20:14:00.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4903281664 bytes 2026-02-19T20:14:00.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 4917972992 bytes 2026-02-19T20:14:00.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 4977840128 bytes 2026-02-19T20:14:00.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 4980744192 bytes 2026-02-19T20:14:00.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5012107264 bytes 2026-02-19T20:14:00.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5037333504 bytes 2026-02-19T20:14:00.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5059182592 bytes 2026-02-19T20:14:00.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5070702592 bytes 2026-02-19T20:14:00.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5102033920 bytes 2026-02-19T20:14:00.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5134825472 bytes 2026-02-19T20:14:00.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5174820864 bytes 2026-02-19T20:14:00.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5236347904 bytes 2026-02-19T20:14:00.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5261204480 bytes 2026-02-19T20:14:00.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5335670784 bytes 2026-02-19T20:14:00.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5382906880 bytes 2026-02-19T20:14:00.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5446650880 bytes 2026-02-19T20:14:00.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5520840704 bytes 2026-02-19T20:14:00.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5521441792 bytes 2026-02-19T20:14:01.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5549167616 bytes 2026-02-19T20:14:01.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5580158976 bytes 2026-02-19T20:14:01.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5618707456 bytes 2026-02-19T20:14:01.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5663521792 bytes 2026-02-19T20:14:01.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5695156224 bytes 2026-02-19T20:14:01.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5738835968 bytes 2026-02-19T20:14:01.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5778259968 bytes 2026-02-19T20:14:01.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 5831888896 bytes 2026-02-19T20:14:01.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 5872424960 bytes 2026-02-19T20:14:01.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 5912608768 bytes 2026-02-19T20:14:01.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 5975016448 bytes 2026-02-19T20:14:01.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6011194368 bytes 2026-02-19T20:14:01.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6048151552 bytes 2026-02-19T20:14:01.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6106382336 bytes 2026-02-19T20:14:01.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6141638656 bytes 2026-02-19T20:14:01.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6146014208 bytes 2026-02-19T20:14:01.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6248320000 bytes 2026-02-19T20:14:01.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6332274688 bytes 2026-02-19T20:14:02.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6408936448 bytes 2026-02-19T20:14:02.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6423582720 bytes 2026-02-19T20:14:02.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6441990144 bytes 2026-02-19T20:14:02.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6483408896 bytes 2026-02-19T20:14:02.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6514015232 bytes 2026-02-19T20:14:02.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6524494848 bytes 2026-02-19T20:14:02.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6572881920 bytes 2026-02-19T20:14:02.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6631819264 bytes 2026-02-19T20:14:02.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6670797824 bytes 2026-02-19T20:14:02.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6704578560 bytes 2026-02-19T20:14:02.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6768541696 bytes 2026-02-19T20:14:02.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6769169408 bytes 2026-02-19T20:14:02.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 6821175296 bytes 2026-02-19T20:14:02.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 6837142528 bytes 2026-02-19T20:14:02.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 6860657664 bytes 2026-02-19T20:14:02.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 6921555968 bytes 2026-02-19T20:14:02.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 6963041280 bytes 2026-02-19T20:14:02.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7058385920 bytes 2026-02-19T20:14:02.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7092416512 bytes 2026-02-19T20:14:03.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7143070720 bytes 2026-02-19T20:14:03.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7199564800 bytes 2026-02-19T20:14:03.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7291621376 bytes 2026-02-19T20:14:03.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7333002240 bytes 2026-02-19T20:14:03.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7341361152 bytes 2026-02-19T20:14:03.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7374738432 bytes 2026-02-19T20:14:03.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7415041024 bytes 2026-02-19T20:14:03.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7453367296 bytes 2026-02-19T20:14:03.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7519954944 bytes 2026-02-19T20:14:03.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7573240832 bytes 2026-02-19T20:14:03.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7585651712 bytes 2026-02-19T20:14:03.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7601382400 bytes 2026-02-19T20:14:03.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7614375936 bytes 2026-02-19T20:14:03.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7661746176 bytes 2026-02-19T20:14:03.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7703819264 bytes 2026-02-19T20:14:03.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 7726860288 bytes 2026-02-19T20:14:03.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 7746476032 bytes 2026-02-19T20:14:03.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 7766643712 bytes 2026-02-19T20:14:03.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 7827221504 bytes 2026-02-19T20:14:04.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 7941502976 bytes 2026-02-19T20:14:04.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 7999671296 bytes 2026-02-19T20:14:04.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8036990976 bytes 2026-02-19T20:14:04.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8058870784 bytes 2026-02-19T20:14:04.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8080734208 bytes 2026-02-19T20:14:04.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8125156352 bytes 2026-02-19T20:14:04.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8178558976 bytes 2026-02-19T20:14:04.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8199166976 bytes 2026-02-19T20:14:04.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8222564352 bytes 2026-02-19T20:14:04.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8267227136 bytes 2026-02-19T20:14:04.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8341499904 bytes 2026-02-19T20:14:04.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 8423079936 bytes 2026-02-19T20:14:04.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 8461521920 bytes 2026-02-19T20:14:04.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 8512769024 bytes 2026-02-19T20:14:04.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 8544200704 bytes 2026-02-19T20:14:04.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 8574198784 bytes 2026-02-19T20:14:04.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 8627267584 bytes 2026-02-19T20:14:04.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 8630622208 bytes 2026-02-19T20:14:05.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 8666942464 bytes 2026-02-19T20:14:05.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 8682788864 bytes 2026-02-19T20:14:05.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 8780725248 bytes 2026-02-19T20:14:05.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 8832944128 bytes 2026-02-19T20:14:05.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 8852695040 bytes 2026-02-19T20:14:05.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 8956435456 bytes 2026-02-19T20:14:05.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 8993631232 bytes 2026-02-19T20:14:05.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 9043068928 bytes 2026-02-19T20:14:05.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 9080955904 bytes 2026-02-19T20:14:05.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:14:05.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:14:05.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:14:05.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:05.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:14:05.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:05.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:05.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:05.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:14:05.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:14:05.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:14:05.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:14:05.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:14:06.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:14:06.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:06.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:06.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:06.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:14:06.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:14:06.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:14:06.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:14:06.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:14:06.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:14:06.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:14:06.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:14:06.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:14:06.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":17880268.8,"bytes_per_snapshot":536084992.0,"last_snapshot_bytes":536408064,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532025,"remote_snapshot_timestamp":1771532025,"replay_state":"idle"} 2026-02-19T20:14:06.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:14:06.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:13:56 2026-02-19T20:14:06.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:14:06.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:14:06.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+stopped 2026-02-19T20:14:06.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: local image is primary 2026-02-19T20:14:06.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:13:56' 2026-02-19T20:14:06.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:14:06.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:14:06.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:14:06.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":17880268.8,"bytes_per_snapshot":536084992.0,"last_snapshot_bytes":536408064,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532025,"remote_snapshot_timestamp":1771532025,"replay_state":"idle"} 2026-02-19T20:14:06.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:14:06.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:14:06.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:14:06.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:14:06.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:14:06.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:14:06.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:14:06.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:14:06.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:13:56' 2026-02-19T20:14:06.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:13:56' 2026-02-19T20:14:06.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:06.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:06.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:06.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:14:06.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:13:56' 2026-02-19T20:14:06.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:14:06.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:14:06.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:14:06.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='OKOKOK142024288mirror.0trial08020.3.0-5336-g802bf111falseOK42054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK44074410mirror.3trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":17880268.8,"bytes_per_snapshot":536084992.0,"last_snapshot_bytes":536408064,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532025,"remote_snapshot_timestamp":1771532025,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:13:56cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3up+stoppedlocal image is primary2026-02-19 20:13:56' 2026-02-19T20:14:06.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:14:06.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:14:06.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:13:56' 2026-02-19T20:14:06.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:14:06.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:14:06.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:14:06.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:14:06.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:14:06.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:06.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:06.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap7 2026-02-19T20:14:06.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap7 2026-02-19T20:14:06.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:14:06.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:06.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:06.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap7 2026-02-19T20:14:06.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:14:34.304 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2026-02-19T20:14:34.304 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2026-02-19T20:14:34.304 INFO:tasks.rbd_mirror_thrash:waiting for 8.187168770417639 secs before reviving daemons 2026-02-19T20:14:34.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:14:34.303+0000 7fe255b09640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 47807) UID: 1000 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:14:35.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:14:36.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:14:36.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:36.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:36.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:36.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:14:36.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:14:36.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:14:36.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:14:36.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:14:36.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:14:36.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:14:36.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:14:36.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:14:36.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:14:36.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:14:36.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:14:36.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:14:36.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:14:36.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:14:36.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:14:36.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:14:36.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:14:36.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2026-02-19T20:14:36.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:14:36.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:14:36.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:14:36.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:14:36.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:14:36.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:14:36.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:14:36.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:14:36.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:14:36.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:14:36.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:14:36.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:14:36.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:14:36.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:14:36.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:14:36.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:14:36.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:14:36.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:14:36.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:14:36.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:14:36.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:14:36.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:14:36.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:14:36.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:14:36.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:14:36.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:14:36.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:14:36.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:14:36.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:14:36.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:14:36.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:14:36.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:14:36.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:36.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:14:38.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 23 2026-02-19T20:14:38.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:14:38.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:14:38.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:14:38.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:14:38.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:14:38.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:38.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:14:38.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:14:38.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:14:38.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:14:38.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:14:38.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:14:38.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:14:38.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:14:38.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:14:38.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:14:38.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:38.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:38.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:14:38.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:14:38.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:14:38.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:14:38.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=23 2026-02-19T20:14:38.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:14:38.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:14:38.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:14:38.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:14:38.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:38.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:38.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:38.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:14:38.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:14:38.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:14:38.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:14:38.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:14:38.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:14:38.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:14:38.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:14:38.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:14:38.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 23 = '' 2026-02-19T20:14:38.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:14:38.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:14:39.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:14:39.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:39.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:39.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:39.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:14:39.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:14:39.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:14:39.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:14:39.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:14:39.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:14:39.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:14:39.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:14:39.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:14:39.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 23 = '' 2026-02-19T20:14:39.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:14:39.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:14:40.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:14:40.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:40.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:40.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:40.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:14:40.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:14:40.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:14:40.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:14:40.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:14:40.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:14:40.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:14:40.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:14:40.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:14:40.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 23 = '' 2026-02-19T20:14:40.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:14:40.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:14:41.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:14:41.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:41.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:41.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:41.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:14:41.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:14:41.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:14:41.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:14:41.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=23 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 23 = 23 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap7 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap7 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:14:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:14:42.491 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2026-02-19T20:14:42.492 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:14:42.492 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2026-02-19T20:14:42.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2026-02-19T20:14:42.492 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2026-02-19T20:14:42.495 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2026-02-19T20:14:42.495 INFO:tasks.rbd_mirror_thrash:waiting for 69.4386651711921 secs before thrashing 2026-02-19T20:14:42.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap7 2026-02-19T20:14:42.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:14:42.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:14:42.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:14:42.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 14 2026-02-19T20:14:42.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:14:42.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:14:42.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:14:42.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:13:51 2026 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:14:42.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:14:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snap6-clean ']' 2026-02-19T20:14:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap6-clean 2026-02-19T20:14:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap6-clean 2026-02-19T20:14:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:14:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:14:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:42.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:42.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:14:43.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:14:43.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:43.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:14:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:43.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:43.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:43.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap7 2026-02-19T20:14:43.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:43.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:43.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap7 2026-02-19T20:14:43.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:14:43.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:14:43.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:43.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:43.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:14:44.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:14:44.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:44.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:14:44.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:45.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:14:45.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:45.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap7-clean 2026-02-19T20:14:45.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap7-clean 2026-02-19T20:14:45.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:14:45.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:45.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:45.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap7-clean 2026-02-19T20:14:45.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:14:45.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:14:45.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:14:45.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:14:45.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:45.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:45.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:14:45.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:14:45.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:14:45.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:14:45.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 72667136 bytes 2026-02-19T20:14:45.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 125155328 bytes 2026-02-19T20:14:45.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 143820800 bytes 2026-02-19T20:14:45.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 180820992 bytes 2026-02-19T20:14:45.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 237845504 bytes 2026-02-19T20:14:45.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 301770752 bytes 2026-02-19T20:14:45.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 346408960 bytes 2026-02-19T20:14:46.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 380553216 bytes 2026-02-19T20:14:46.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 418680832 bytes 2026-02-19T20:14:46.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 436691968 bytes 2026-02-19T20:14:46.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 438607872 bytes 2026-02-19T20:14:46.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 508279808 bytes 2026-02-19T20:14:46.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 568550400 bytes 2026-02-19T20:14:46.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 613446656 bytes 2026-02-19T20:14:46.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 648373248 bytes 2026-02-19T20:14:46.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 664138752 bytes 2026-02-19T20:14:46.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 694258688 bytes 2026-02-19T20:14:46.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 725370880 bytes 2026-02-19T20:14:46.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 769059840 bytes 2026-02-19T20:14:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 819013632 bytes 2026-02-19T20:14:46.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 881670144 bytes 2026-02-19T20:14:46.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 897514496 bytes 2026-02-19T20:14:46.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 986227712 bytes 2026-02-19T20:14:46.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1022558208 bytes 2026-02-19T20:14:46.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1059978240 bytes 2026-02-19T20:14:47.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1092156416 bytes 2026-02-19T20:14:47.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1107961856 bytes 2026-02-19T20:14:47.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1127139328 bytes 2026-02-19T20:14:47.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1262839808 bytes 2026-02-19T20:14:47.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1309840384 bytes 2026-02-19T20:14:47.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1371408384 bytes 2026-02-19T20:14:47.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1384915968 bytes 2026-02-19T20:14:47.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1461574656 bytes 2026-02-19T20:14:47.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1541956608 bytes 2026-02-19T20:14:47.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1542568960 bytes 2026-02-19T20:14:47.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1612574720 bytes 2026-02-19T20:14:47.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1616246784 bytes 2026-02-19T20:14:47.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1647880192 bytes 2026-02-19T20:14:47.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1710569472 bytes 2026-02-19T20:14:47.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1740020736 bytes 2026-02-19T20:14:47.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1791684608 bytes 2026-02-19T20:14:47.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1818255360 bytes 2026-02-19T20:14:48.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1871299584 bytes 2026-02-19T20:14:48.056 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1895192576 bytes 2026-02-19T20:14:48.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1954332672 bytes 2026-02-19T20:14:48.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1993170944 bytes 2026-02-19T20:14:48.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2003435520 bytes 2026-02-19T20:14:48.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2086856704 bytes 2026-02-19T20:14:48.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2098840576 bytes 2026-02-19T20:14:48.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2138809344 bytes 2026-02-19T20:14:48.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2217298944 bytes 2026-02-19T20:14:48.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2258499584 bytes 2026-02-19T20:14:48.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2305545216 bytes 2026-02-19T20:14:48.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2318689280 bytes 2026-02-19T20:14:48.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2392895488 bytes 2026-02-19T20:14:48.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2430901248 bytes 2026-02-19T20:14:48.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2447114240 bytes 2026-02-19T20:14:48.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2496379904 bytes 2026-02-19T20:14:48.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2507678720 bytes 2026-02-19T20:14:48.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2541733888 bytes 2026-02-19T20:14:49.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2656078848 bytes 2026-02-19T20:14:49.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2700645376 bytes 2026-02-19T20:14:49.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2710116352 bytes 2026-02-19T20:14:49.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2751494144 bytes 2026-02-19T20:14:49.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2822862848 bytes 2026-02-19T20:14:49.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2865099776 bytes 2026-02-19T20:14:49.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2903839744 bytes 2026-02-19T20:14:49.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2944532480 bytes 2026-02-19T20:14:49.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2973293568 bytes 2026-02-19T20:14:49.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3084205056 bytes 2026-02-19T20:14:49.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3127069696 bytes 2026-02-19T20:14:49.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3204748288 bytes 2026-02-19T20:14:49.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3285929984 bytes 2026-02-19T20:14:49.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3338577920 bytes 2026-02-19T20:14:49.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3357090816 bytes 2026-02-19T20:14:49.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3394481152 bytes 2026-02-19T20:14:49.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3412804608 bytes 2026-02-19T20:14:50.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3470131200 bytes 2026-02-19T20:14:50.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3470734336 bytes 2026-02-19T20:14:50.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3529800704 bytes 2026-02-19T20:14:50.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3549171712 bytes 2026-02-19T20:14:50.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3582277632 bytes 2026-02-19T20:14:50.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3602792448 bytes 2026-02-19T20:14:50.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3643559936 bytes 2026-02-19T20:14:50.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3722857472 bytes 2026-02-19T20:14:50.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3752664064 bytes 2026-02-19T20:14:50.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3813647360 bytes 2026-02-19T20:14:50.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3847542784 bytes 2026-02-19T20:14:50.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3896485888 bytes 2026-02-19T20:14:50.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3920293888 bytes 2026-02-19T20:14:50.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3949374464 bytes 2026-02-19T20:14:50.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4050219008 bytes 2026-02-19T20:14:50.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4072733696 bytes 2026-02-19T20:14:50.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4137691136 bytes 2026-02-19T20:14:50.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4181126144 bytes 2026-02-19T20:14:51.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4233070592 bytes 2026-02-19T20:14:51.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4276509696 bytes 2026-02-19T20:14:51.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4324635648 bytes 2026-02-19T20:14:51.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4408724480 bytes 2026-02-19T20:14:51.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4463356928 bytes 2026-02-19T20:14:51.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4487027712 bytes 2026-02-19T20:14:51.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4577376256 bytes 2026-02-19T20:14:51.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4639354880 bytes 2026-02-19T20:14:51.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4661378048 bytes 2026-02-19T20:14:51.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4674716672 bytes 2026-02-19T20:14:51.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4692705280 bytes 2026-02-19T20:14:51.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4727127040 bytes 2026-02-19T20:14:51.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4765609984 bytes 2026-02-19T20:14:52.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4847516672 bytes 2026-02-19T20:14:52.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4870706176 bytes 2026-02-19T20:14:52.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4933803008 bytes 2026-02-19T20:14:52.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4975271936 bytes 2026-02-19T20:14:52.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5012341760 bytes 2026-02-19T20:14:52.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5078169600 bytes 2026-02-19T20:14:52.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5120734208 bytes 2026-02-19T20:14:52.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5156617216 bytes 2026-02-19T20:14:52.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5167734784 bytes 2026-02-19T20:14:52.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5211225088 bytes 2026-02-19T20:14:52.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5211828224 bytes 2026-02-19T20:14:52.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5282408448 bytes 2026-02-19T20:14:52.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5344425984 bytes 2026-02-19T20:14:52.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5374239744 bytes 2026-02-19T20:14:52.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5439540224 bytes 2026-02-19T20:14:52.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5473747968 bytes 2026-02-19T20:14:52.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5503917056 bytes 2026-02-19T20:14:53.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5587765248 bytes 2026-02-19T20:14:53.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5625292800 bytes 2026-02-19T20:14:53.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5690241024 bytes 2026-02-19T20:14:53.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5694628864 bytes 2026-02-19T20:14:53.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5743815680 bytes 2026-02-19T20:14:53.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5833771008 bytes 2026-02-19T20:14:53.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5865644032 bytes 2026-02-19T20:14:53.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5904446464 bytes 2026-02-19T20:14:53.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5940935680 bytes 2026-02-19T20:14:53.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5956032512 bytes 2026-02-19T20:14:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5996607488 bytes 2026-02-19T20:14:53.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6096971776 bytes 2026-02-19T20:14:53.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6117131264 bytes 2026-02-19T20:14:53.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6146801664 bytes 2026-02-19T20:14:53.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6181031936 bytes 2026-02-19T20:14:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6213429248 bytes 2026-02-19T20:14:53.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6225614848 bytes 2026-02-19T20:14:53.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6238923776 bytes 2026-02-19T20:14:54.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6293163008 bytes 2026-02-19T20:14:54.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6315075584 bytes 2026-02-19T20:14:54.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6378207232 bytes 2026-02-19T20:14:54.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6418752512 bytes 2026-02-19T20:14:54.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6488688640 bytes 2026-02-19T20:14:54.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6564689920 bytes 2026-02-19T20:14:54.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6585482240 bytes 2026-02-19T20:14:54.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6655000576 bytes 2026-02-19T20:14:54.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6661171200 bytes 2026-02-19T20:14:54.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6734150656 bytes 2026-02-19T20:14:54.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6736370688 bytes 2026-02-19T20:14:54.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6795806720 bytes 2026-02-19T20:14:54.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6801473536 bytes 2026-02-19T20:14:54.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6822255616 bytes 2026-02-19T20:14:54.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6822859776 bytes 2026-02-19T20:14:54.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6895599616 bytes 2026-02-19T20:14:54.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6962196480 bytes 2026-02-19T20:14:54.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7039682560 bytes 2026-02-19T20:14:55.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7078158336 bytes 2026-02-19T20:14:55.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7180321792 bytes 2026-02-19T20:14:55.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7226562560 bytes 2026-02-19T20:14:55.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7287492608 bytes 2026-02-19T20:14:55.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7406870528 bytes 2026-02-19T20:14:55.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7435187200 bytes 2026-02-19T20:14:55.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7451386880 bytes 2026-02-19T20:14:55.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7495094272 bytes 2026-02-19T20:14:55.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:14:55.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:14:55.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:14:55.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:55.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:14:55.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:55.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:55.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:55.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:14:55.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:14:55.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:14:55.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:14:55.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:14:56.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:14:56.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:56.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:56.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:56.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:14:56.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:14:56.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:14:56.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:14:56.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:14:56.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:14:56.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:14:56.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:14:56.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:14:56.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":17862348.8,"bytes_per_snapshot":536139264.0,"last_snapshot_bytes":535870464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532077,"remote_snapshot_timestamp":1771532077,"replay_state":"idle"} 2026-02-19T20:14:56.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:14:56.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:14:56 2026-02-19T20:14:56.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:14:56.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:14:56.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+stopped 2026-02-19T20:14:56.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: local image is primary 2026-02-19T20:14:56.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:14:56' 2026-02-19T20:14:56.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:14:56.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:14:56.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:14:56.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":17862348.8,"bytes_per_snapshot":536139264.0,"last_snapshot_bytes":535870464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532077,"remote_snapshot_timestamp":1771532077,"replay_state":"idle"} 2026-02-19T20:14:56.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:14:56.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:14:56.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:14:56.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:14:56.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:14:56.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:14:56.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:14:56.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:14:56.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:14:56' 2026-02-19T20:14:56.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:14:56' 2026-02-19T20:14:56.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:14:56.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:56.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:56.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:14:56.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:14:56' 2026-02-19T20:14:56.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:14:56.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:14:56.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:14:56.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='OKOKOK142054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK44074410mirror.3trial08020.3.0-5336-g802bf111falseOK45044507mirror.0trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":17862348.8,"bytes_per_snapshot":536139264.0,"last_snapshot_bytes":535870464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532077,"remote_snapshot_timestamp":1771532077,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:14:56cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3up+stoppedlocal image is primary2026-02-19 20:14:56' 2026-02-19T20:14:56.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:14:56.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:14:56.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:14:56' 2026-02-19T20:14:56.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:14:56.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:14:56.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:14:56.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:14:56.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:14:56.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:56.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:14:56.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap8 2026-02-19T20:14:56.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap8 2026-02-19T20:14:56.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:14:56.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:14:56.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:14:56.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap8 2026-02-19T20:14:56.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:15:00.840 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7690371719294411) > weight (0.25) 2026-02-19T20:15:00.840 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8852893631355575) > weight (0.25) 2026-02-19T20:15:00.840 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.57776043788711) > weight (0.25) 2026-02-19T20:15:00.840 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9856915267244627) > weight (0.25) 2026-02-19T20:15:00.841 INFO:tasks.rbd_mirror_thrash:waiting for 90.50330423046577 secs before thrashing 2026-02-19T20:15:25.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:15:25.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:25.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:25.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:25.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:15:25.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:25.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:25.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:25.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:15:25.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:15:25.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:15:25.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:15:26.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:15:26.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:26.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:26.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:26.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:15:26.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:15:26.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:15:26.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:15:26.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:15:26.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:15:26.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:15:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:15:26.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:15:26.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:15:26.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:15:26.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:15:26.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:15:26.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:15:26.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:15:26.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:15:26.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:15:26.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:15:26.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:15:26.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:15:26.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:15:26.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:15:26.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:15:26.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:15:26.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:15:26.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:15:26.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:15:26.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:15:26.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:15:26.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:15:26.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:15:26.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:15:26.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:15:26.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:15:26.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:15:26.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:15:26.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:15:26.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:15:26.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:15:26.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:15:26.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:15:26.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:15:26.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:15:26.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:15:26.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:15:26.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:15:26.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:15:26.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:15:26.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:15:26.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:15:26.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:15:26.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:15:26.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:15:26.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:26.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:15:28.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 26 2026-02-19T20:15:28.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:15:28.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:15:28.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:15:28.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:15:28.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:15:28.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:28.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:15:28.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:15:28.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:15:28.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:15:28.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:15:28.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:15:28.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:15:28.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:15:28.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:15:28.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:15:28.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:28.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:28.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:15:28.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:15:28.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:15:28.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:15:28.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=26 2026-02-19T20:15:28.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:15:28.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:15:28.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:15:28.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:15:28.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:28.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:28.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:28.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:15:28.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:15:28.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:15:28.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:15:28.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:15:28.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:15:28.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:15:28.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:15:28.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:15:28.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 26 = '' 2026-02-19T20:15:28.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:15:28.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:15:29.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:15:29.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:29.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:29.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:29.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:15:29.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:15:29.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:15:29.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:15:29.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:15:29.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:15:29.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:15:29.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:15:29.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:15:29.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 26 = '' 2026-02-19T20:15:29.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:15:29.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:15:30.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:15:30.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:30.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:30.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:30.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:15:30.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:15:30.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:15:30.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:15:30.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:15:30.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:15:30.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:15:30.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:15:30.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:15:30.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 26 = '' 2026-02-19T20:15:30.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:15:30.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:15:31.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:15:31.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:31.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:31.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:31.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:15:31.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:15:31.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:15:31.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:15:31.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=26 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 26 = 26 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap8 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap8 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:15:31.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:15:32.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap8 2026-02-19T20:15:32.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 16 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:15:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:15:32.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:13:51 2026 2026-02-19T20:15:32.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:15:32.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:15:32.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:15:32.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:15:32.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:15:32.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:15:32.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:15:32.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snap7-clean ']' 2026-02-19T20:15:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap7-clean 2026-02-19T20:15:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap7-clean 2026-02-19T20:15:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:15:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:15:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:15:33.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:15:33.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:33.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:15:33.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:33.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:15:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap8 2026-02-19T20:15:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap8 2026-02-19T20:15:33.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:15:33.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:15:33.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:33.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:33.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:15:34.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:15:34.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:34.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:15:34.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:35.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:15:35.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:15:35.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap8-clean 2026-02-19T20:15:35.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap8-clean 2026-02-19T20:15:35.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:15:35.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:35.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:35.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap8-clean 2026-02-19T20:15:35.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:15:35.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:15:35.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:15:35.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:15:35.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:35.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:35.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:15:35.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:15:35.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:15:35.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:15:35.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 73921536 bytes 2026-02-19T20:15:35.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 133828608 bytes 2026-02-19T20:15:36.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 221050880 bytes 2026-02-19T20:15:36.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 257202176 bytes 2026-02-19T20:15:36.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 358295552 bytes 2026-02-19T20:15:36.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 468374528 bytes 2026-02-19T20:15:36.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 502753280 bytes 2026-02-19T20:15:36.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 544188416 bytes 2026-02-19T20:15:36.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 604917760 bytes 2026-02-19T20:15:36.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 613417984 bytes 2026-02-19T20:15:36.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 680932352 bytes 2026-02-19T20:15:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 716555264 bytes 2026-02-19T20:15:36.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 762626048 bytes 2026-02-19T20:15:36.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 860627968 bytes 2026-02-19T20:15:36.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 914641920 bytes 2026-02-19T20:15:36.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 955417600 bytes 2026-02-19T20:15:36.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 974101504 bytes 2026-02-19T20:15:37.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 1031292928 bytes 2026-02-19T20:15:37.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 1078747136 bytes 2026-02-19T20:15:37.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 1099821056 bytes 2026-02-19T20:15:37.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1110364160 bytes 2026-02-19T20:15:37.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1177547776 bytes 2026-02-19T20:15:37.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1202398208 bytes 2026-02-19T20:15:37.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1312861184 bytes 2026-02-19T20:15:37.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1352326144 bytes 2026-02-19T20:15:37.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1352932352 bytes 2026-02-19T20:15:37.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1358772224 bytes 2026-02-19T20:15:37.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1409967104 bytes 2026-02-19T20:15:37.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1473005568 bytes 2026-02-19T20:15:37.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1495448576 bytes 2026-02-19T20:15:37.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1515322368 bytes 2026-02-19T20:15:37.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1552109568 bytes 2026-02-19T20:15:37.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1640572928 bytes 2026-02-19T20:15:37.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1668230144 bytes 2026-02-19T20:15:37.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1688769536 bytes 2026-02-19T20:15:37.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1702689792 bytes 2026-02-19T20:15:37.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1738422272 bytes 2026-02-19T20:15:38.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1780105216 bytes 2026-02-19T20:15:38.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1882619904 bytes 2026-02-19T20:15:38.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1896375296 bytes 2026-02-19T20:15:38.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1936457728 bytes 2026-02-19T20:15:38.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1981275136 bytes 2026-02-19T20:15:38.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1986091008 bytes 2026-02-19T20:15:38.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2098354176 bytes 2026-02-19T20:15:38.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2121088000 bytes 2026-02-19T20:15:38.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2219445248 bytes 2026-02-19T20:15:38.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2222879744 bytes 2026-02-19T20:15:38.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2271283200 bytes 2026-02-19T20:15:38.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2295201792 bytes 2026-02-19T20:15:38.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2295810048 bytes 2026-02-19T20:15:38.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2320497664 bytes 2026-02-19T20:15:38.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2342501376 bytes 2026-02-19T20:15:38.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2376461312 bytes 2026-02-19T20:15:38.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2414204928 bytes 2026-02-19T20:15:38.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2489946112 bytes 2026-02-19T20:15:38.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2508363776 bytes 2026-02-19T20:15:39.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2617067520 bytes 2026-02-19T20:15:39.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2617695232 bytes 2026-02-19T20:15:39.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2666192896 bytes 2026-02-19T20:15:39.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2711865344 bytes 2026-02-19T20:15:39.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2781979648 bytes 2026-02-19T20:15:39.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2866149376 bytes 2026-02-19T20:15:39.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2907340800 bytes 2026-02-19T20:15:39.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2922883072 bytes 2026-02-19T20:15:39.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3003342848 bytes 2026-02-19T20:15:39.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3003988992 bytes 2026-02-19T20:15:39.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3064764416 bytes 2026-02-19T20:15:39.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3086422016 bytes 2026-02-19T20:15:39.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3121440768 bytes 2026-02-19T20:15:39.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3173481472 bytes 2026-02-19T20:15:39.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3274419200 bytes 2026-02-19T20:15:40.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3376037888 bytes 2026-02-19T20:15:40.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3440484352 bytes 2026-02-19T20:15:40.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3449479168 bytes 2026-02-19T20:15:40.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3459971072 bytes 2026-02-19T20:15:40.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3512232960 bytes 2026-02-19T20:15:40.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3590340608 bytes 2026-02-19T20:15:40.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3602733056 bytes 2026-02-19T20:15:40.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3694532608 bytes 2026-02-19T20:15:40.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3721701376 bytes 2026-02-19T20:15:40.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3773773824 bytes 2026-02-19T20:15:40.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3813444608 bytes 2026-02-19T20:15:40.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3830560768 bytes 2026-02-19T20:15:40.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3875312640 bytes 2026-02-19T20:15:40.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3926427648 bytes 2026-02-19T20:15:40.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3927046144 bytes 2026-02-19T20:15:40.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3961731072 bytes 2026-02-19T20:15:40.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3998857216 bytes 2026-02-19T20:15:40.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4062855168 bytes 2026-02-19T20:15:40.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4118077440 bytes 2026-02-19T20:15:40.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4118705152 bytes 2026-02-19T20:15:41.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4235450368 bytes 2026-02-19T20:15:41.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4275028992 bytes 2026-02-19T20:15:41.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4299623424 bytes 2026-02-19T20:15:41.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4349607936 bytes 2026-02-19T20:15:41.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4408290304 bytes 2026-02-19T20:15:41.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4427400192 bytes 2026-02-19T20:15:41.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4433908736 bytes 2026-02-19T20:15:41.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4460271616 bytes 2026-02-19T20:15:41.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4511304704 bytes 2026-02-19T20:15:41.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4556603392 bytes 2026-02-19T20:15:41.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4614947840 bytes 2026-02-19T20:15:41.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4659364864 bytes 2026-02-19T20:15:41.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4690443264 bytes 2026-02-19T20:15:41.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4733516800 bytes 2026-02-19T20:15:41.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4821440512 bytes 2026-02-19T20:15:41.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4854171648 bytes 2026-02-19T20:15:41.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4880060416 bytes 2026-02-19T20:15:41.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4914523136 bytes 2026-02-19T20:15:42.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4929928192 bytes 2026-02-19T20:15:42.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4970699776 bytes 2026-02-19T20:15:42.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5027637248 bytes 2026-02-19T20:15:42.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5077083136 bytes 2026-02-19T20:15:42.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5113772032 bytes 2026-02-19T20:15:42.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5150080000 bytes 2026-02-19T20:15:42.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5170762752 bytes 2026-02-19T20:15:42.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5200762880 bytes 2026-02-19T20:15:42.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5272072192 bytes 2026-02-19T20:15:42.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5287181312 bytes 2026-02-19T20:15:42.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5296449536 bytes 2026-02-19T20:15:42.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5326294016 bytes 2026-02-19T20:15:42.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5339028480 bytes 2026-02-19T20:15:42.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5375097856 bytes 2026-02-19T20:15:42.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5404101632 bytes 2026-02-19T20:15:42.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5461473280 bytes 2026-02-19T20:15:42.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5464209408 bytes 2026-02-19T20:15:42.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5540836352 bytes 2026-02-19T20:15:42.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5564154880 bytes 2026-02-19T20:15:43.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5591093248 bytes 2026-02-19T20:15:43.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5591724032 bytes 2026-02-19T20:15:43.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5660805120 bytes 2026-02-19T20:15:43.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5685517312 bytes 2026-02-19T20:15:43.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5724814336 bytes 2026-02-19T20:15:43.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5769566208 bytes 2026-02-19T20:15:43.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5812664320 bytes 2026-02-19T20:15:43.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5844714496 bytes 2026-02-19T20:15:43.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5915738112 bytes 2026-02-19T20:15:43.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5962362880 bytes 2026-02-19T20:15:43.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5968172032 bytes 2026-02-19T20:15:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6015722496 bytes 2026-02-19T20:15:43.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6054011904 bytes 2026-02-19T20:15:43.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6082164736 bytes 2026-02-19T20:15:43.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6139990016 bytes 2026-02-19T20:15:43.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6153431040 bytes 2026-02-19T20:15:43.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6174472192 bytes 2026-02-19T20:15:43.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6256841728 bytes 2026-02-19T20:15:43.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6277345280 bytes 2026-02-19T20:15:44.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6303268864 bytes 2026-02-19T20:15:44.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6387887104 bytes 2026-02-19T20:15:44.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6412484608 bytes 2026-02-19T20:15:44.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6428028928 bytes 2026-02-19T20:15:44.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6509219840 bytes 2026-02-19T20:15:44.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6551824384 bytes 2026-02-19T20:15:44.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6622632960 bytes 2026-02-19T20:15:44.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6658061312 bytes 2026-02-19T20:15:44.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6701419520 bytes 2026-02-19T20:15:44.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6728767488 bytes 2026-02-19T20:15:44.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6736994304 bytes 2026-02-19T20:15:44.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6790173696 bytes 2026-02-19T20:15:44.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6827853824 bytes 2026-02-19T20:15:44.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6835477504 bytes 2026-02-19T20:15:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6901520384 bytes 2026-02-19T20:15:44.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6924981248 bytes 2026-02-19T20:15:44.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6953503744 bytes 2026-02-19T20:15:44.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6995125248 bytes 2026-02-19T20:15:44.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7029527552 bytes 2026-02-19T20:15:45.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7056246784 bytes 2026-02-19T20:15:45.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7065748480 bytes 2026-02-19T20:15:45.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7110710272 bytes 2026-02-19T20:15:45.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7136251904 bytes 2026-02-19T20:15:45.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7144195072 bytes 2026-02-19T20:15:45.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7202202624 bytes 2026-02-19T20:15:45.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7240914944 bytes 2026-02-19T20:15:45.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7269629952 bytes 2026-02-19T20:15:45.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7323471872 bytes 2026-02-19T20:15:45.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7375273984 bytes 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:15:45.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:15:46.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:15:46.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:46.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:46.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:46.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:15:46.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:15:46.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:15:46.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:15:46.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:15:46.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:15:46.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:15:46.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:15:46.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:15:46.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":536139264.0,"last_snapshot_bytes":535870464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532077,"remote_snapshot_timestamp":1771532077,"replay_state":"idle"} 2026-02-19T20:15:46.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:15:46.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:15:26 2026-02-19T20:15:46.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:15:46.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:15:46.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+stopped 2026-02-19T20:15:46.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: local image is primary 2026-02-19T20:15:46.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:15:26' 2026-02-19T20:15:46.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:15:46.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:15:46.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:15:46.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":536139264.0,"last_snapshot_bytes":535870464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532077,"remote_snapshot_timestamp":1771532077,"replay_state":"idle"} 2026-02-19T20:15:46.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:15:46.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:15:46.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:15:46.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:15:46.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:15:46.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:15:46.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:15:46.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:15:46.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:15:26' 2026-02-19T20:15:46.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:15:26' 2026-02-19T20:15:46.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:15:46.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:46.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:46.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:15:46.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:15:26' 2026-02-19T20:15:46.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:15:46.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:15:46.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:15:46.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='OKOKOK142054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK44074410mirror.3trial08020.3.0-5336-g802bf111falseOK45044507mirror.0trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":0.0,"bytes_per_snapshot":536139264.0,"last_snapshot_bytes":535870464,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532077,"remote_snapshot_timestamp":1771532077,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:15:26cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3up+stoppedlocal image is primary2026-02-19 20:15:26' 2026-02-19T20:15:46.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:15:46.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:15:46.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:15:26' 2026-02-19T20:15:46.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:15:46.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:15:46.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:15:46.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:15:46.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:15:46.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:15:46.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:15:46.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap9 2026-02-19T20:15:46.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap9 2026-02-19T20:15:46.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:15:46.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:15:46.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:15:46.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap9 2026-02-19T20:15:46.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:15:51.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4766742805359061) > weight (0.25) 2026-02-19T20:15:51.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9648966029126437) > weight (0.25) 2026-02-19T20:15:51.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8898613403517615) > weight (0.25) 2026-02-19T20:15:51.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7606270870681593) > weight (0.25) 2026-02-19T20:15:51.935 INFO:tasks.rbd_mirror_thrash:waiting for 62.43578460494956 secs before thrashing 2026-02-19T20:16:15.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:16:15.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:15.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:15.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:15.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:16:15.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:15.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:15.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:15.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:16:15.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:16:15.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:16:15.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:16:16.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:16:16.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:16.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:16.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:16.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:16:16.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:16:16.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:16:16.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:16:16.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:16:16.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:16:16.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:16:16.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:16:16.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:16:16.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:16:16.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:16:16.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:16:16.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:16:16.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:16:16.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:16:16.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:16:16.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:16:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:16:16.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:16:16.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:16:16.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:16:16.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:16:16.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:16:16.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:16:16.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:16:16.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:16:16.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:16:16.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:16:16.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:16:16.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:16:16.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:16:16.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:16:16.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:16:16.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:16:16.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:16:16.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:16:16.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:16:16.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:16:16.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:16:16.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:16:16.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:16:16.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:16:16.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:16:16.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:16:16.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:16:16.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:16:16.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:16:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:16:16.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:16:16.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:16:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:16:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:16:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:16:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:16:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:16:16.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:16:16.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:16:16.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:16:16.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:16:16.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:16:16.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:16:16.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:16:16.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:16:16.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:16:16.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:16:16.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:16:16.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:16:16.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:16:16.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:16:16.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:16:16.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:16:16.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:16:16.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:16:16.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:16:18.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 29 2026-02-19T20:16:18.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:16:18.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:16:18.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:16:18.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:16:18.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:16:18.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:18.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:16:18.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:16:18.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:16:18.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:16:18.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:16:18.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:16:18.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:16:18.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:16:18.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:16:18.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:16:18.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:18.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:18.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:16:18.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:16:18.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:16:18.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:16:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=29 2026-02-19T20:16:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:16:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:16:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:16:18.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:16:18.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:18.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:18.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:18.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:16:18.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:16:18.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:16:18.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:16:18.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:16:18.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:16:18.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:16:18.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:16:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:16:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 29 = '' 2026-02-19T20:16:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:16:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:16:19.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:16:19.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:19.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:19.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:19.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:16:19.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:16:19.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:16:19.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:16:19.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:16:19.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:16:19.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:16:19.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:16:19.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:16:19.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 29 = '' 2026-02-19T20:16:19.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:16:19.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:16:20.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:16:20.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:20.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:20.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:20.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:16:20.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:16:20.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:16:20.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:16:20.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:16:20.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:16:20.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:16:20.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:16:20.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:16:20.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 29 = '' 2026-02-19T20:16:20.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:16:20.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:16:21.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:16:21.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:21.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:21.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:21.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:16:21.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:16:21.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:16:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:16:21.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=29 2026-02-19T20:16:21.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 29 = 29 2026-02-19T20:16:21.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:16:21.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap9 2026-02-19T20:16:21.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:21.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:21.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:21.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap9 2026-02-19T20:16:21.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:16:21.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:16:21.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:16:22.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap9 2026-02-19T20:16:22.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 18 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:16:22.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:15:33 2026 2026-02-19T20:16:22.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:16:22.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:16:22.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:16:22.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:16:22.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:16:22.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:16:22.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:16:22.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snap8-clean ']' 2026-02-19T20:16:22.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap8-clean 2026-02-19T20:16:22.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:22.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:22.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap8-clean 2026-02-19T20:16:22.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:16:22.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:16:22.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:22.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:22.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:16:23.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:16:23.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:23.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:16:23.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:23.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:23.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:16:23.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap9 2026-02-19T20:16:23.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:23.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:23.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap9 2026-02-19T20:16:23.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:16:23.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:16:23.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:23.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:23.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:16:24.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:16:24.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:24.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:16:24.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:25.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:16:25.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:16:25.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap9-clean 2026-02-19T20:16:25.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap9-clean 2026-02-19T20:16:25.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:16:25.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:25.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:25.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap9-clean 2026-02-19T20:16:25.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:16:25.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:16:25.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror test 2026-02-19T20:16:25.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:16:25.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:25.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:25.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:16:25.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=5 2026-02-19T20:16:25.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:16:25.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2026-02-19T20:16:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 21309440 bytes 2026-02-19T20:16:25.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 21922816 bytes 2026-02-19T20:16:25.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 31150080 bytes 2026-02-19T20:16:25.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 51512320 bytes 2026-02-19T20:16:25.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 62998528 bytes 2026-02-19T20:16:25.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 98830336 bytes 2026-02-19T20:16:25.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 126502912 bytes 2026-02-19T20:16:25.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 196057088 bytes 2026-02-19T20:16:26.056 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 226788352 bytes 2026-02-19T20:16:26.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 355884032 bytes 2026-02-19T20:16:26.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 410270720 bytes 2026-02-19T20:16:26.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 439634944 bytes 2026-02-19T20:16:26.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 499826688 bytes 2026-02-19T20:16:26.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 556426240 bytes 2026-02-19T20:16:26.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 557000704 bytes 2026-02-19T20:16:26.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 593081344 bytes 2026-02-19T20:16:26.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 639186944 bytes 2026-02-19T20:16:26.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 654074880 bytes 2026-02-19T20:16:26.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 691146752 bytes 2026-02-19T20:16:26.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 763920384 bytes 2026-02-19T20:16:26.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 797929472 bytes 2026-02-19T20:16:26.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 909843456 bytes 2026-02-19T20:16:26.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 984340480 bytes 2026-02-19T20:16:27.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1013298176 bytes 2026-02-19T20:16:27.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1026826240 bytes 2026-02-19T20:16:27.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1031608320 bytes 2026-02-19T20:16:27.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1045400576 bytes 2026-02-19T20:16:27.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1118173184 bytes 2026-02-19T20:16:27.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1160035328 bytes 2026-02-19T20:16:27.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1217345536 bytes 2026-02-19T20:16:27.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1278286848 bytes 2026-02-19T20:16:27.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1304249344 bytes 2026-02-19T20:16:27.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1356350464 bytes 2026-02-19T20:16:27.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1372563456 bytes 2026-02-19T20:16:27.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1408367616 bytes 2026-02-19T20:16:27.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1449537536 bytes 2026-02-19T20:16:27.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1476123648 bytes 2026-02-19T20:16:27.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1544977408 bytes 2026-02-19T20:16:27.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1579862016 bytes 2026-02-19T20:16:27.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1663972352 bytes 2026-02-19T20:16:27.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1716006912 bytes 2026-02-19T20:16:28.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1762573312 bytes 2026-02-19T20:16:28.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1767889920 bytes 2026-02-19T20:16:28.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1797573632 bytes 2026-02-19T20:16:28.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1823817728 bytes 2026-02-19T20:16:28.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1829898240 bytes 2026-02-19T20:16:28.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1854604288 bytes 2026-02-19T20:16:28.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1941421056 bytes 2026-02-19T20:16:28.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1964078080 bytes 2026-02-19T20:16:28.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2022100992 bytes 2026-02-19T20:16:28.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2045283328 bytes 2026-02-19T20:16:28.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2097518592 bytes 2026-02-19T20:16:28.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2140633088 bytes 2026-02-19T20:16:28.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2167891968 bytes 2026-02-19T20:16:28.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2208646144 bytes 2026-02-19T20:16:28.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2256906240 bytes 2026-02-19T20:16:29.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2308776960 bytes 2026-02-19T20:16:29.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2320493568 bytes 2026-02-19T20:16:29.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2345338880 bytes 2026-02-19T20:16:29.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2411586560 bytes 2026-02-19T20:16:29.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2412188672 bytes 2026-02-19T20:16:29.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2450830336 bytes 2026-02-19T20:16:29.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2493256704 bytes 2026-02-19T20:16:29.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2505509888 bytes 2026-02-19T20:16:29.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2551081984 bytes 2026-02-19T20:16:29.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2587703296 bytes 2026-02-19T20:16:29.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2688313344 bytes 2026-02-19T20:16:29.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2767903744 bytes 2026-02-19T20:16:29.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2803753984 bytes 2026-02-19T20:16:29.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2844997632 bytes 2026-02-19T20:16:29.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2868012032 bytes 2026-02-19T20:16:29.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2887818240 bytes 2026-02-19T20:16:29.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2904314880 bytes 2026-02-19T20:16:29.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2960999424 bytes 2026-02-19T20:16:30.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2977679360 bytes 2026-02-19T20:16:30.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3040011264 bytes 2026-02-19T20:16:30.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3066181632 bytes 2026-02-19T20:16:30.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3130808320 bytes 2026-02-19T20:16:30.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3178308608 bytes 2026-02-19T20:16:30.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3179755520 bytes 2026-02-19T20:16:30.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3236291584 bytes 2026-02-19T20:16:30.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3282117632 bytes 2026-02-19T20:16:30.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3372209152 bytes 2026-02-19T20:16:30.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:16:30.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:16:30.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:16:31.344 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38241150939965063) > weight (0.25) 2026-02-19T20:16:31.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3120148306939957) > weight (0.25) 2026-02-19T20:16:31.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.321877928538156) > weight (0.25) 2026-02-19T20:16:31.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8950314429873595) > weight (0.25) 2026-02-19T20:16:31.345 INFO:tasks.rbd_mirror_thrash:waiting for 111.47473292278224 secs before thrashing 2026-02-19T20:16:31.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying '' '' 2026-02-19T20:16:31.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:31.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:31.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:31.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:16:31.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local description_pattern= 2026-02-19T20:16:31.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local service_pattern= 2026-02-19T20:16:31.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:16:31.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:16:31.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror image status mirror/test 2026-02-19T20:16:31.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='test: 2026-02-19T20:16:31.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: global_id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:16:31.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+replaying 2026-02-19T20:16:31.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: replaying, {"bytes_per_second":17878391.47,"bytes_per_snapshot":536568832.0,"last_snapshot_bytes":536351744,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532177,"remote_snapshot_timestamp":1771532177,"replay_state":"idle"} 2026-02-19T20:16:31.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: service: mirror.2 on trial080 2026-02-19T20:16:31.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:16:26 2026-02-19T20:16:31.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: peer_sites: 2026-02-19T20:16:31.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: name: cluster2 2026-02-19T20:16:31.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: state: up+stopped 2026-02-19T20:16:31.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: description: local image is primary 2026-02-19T20:16:31.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: last_update: 2026-02-19 20:16:26' 2026-02-19T20:16:31.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ state: .*up+replaying' 2026-02-19T20:16:31.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: state: up+replaying 2026-02-19T20:16:31.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^ description: .*' 2026-02-19T20:16:31.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: description: replaying, {"bytes_per_second":17878391.47,"bytes_per_snapshot":536568832.0,"last_snapshot_bytes":536351744,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532177,"remote_snapshot_timestamp":1771532177,"replay_state":"idle"} 2026-02-19T20:16:31.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:16:31.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:16:31.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep '^up+' 2026-02-19T20:16:31.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:16:31.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep 'service: *mirror..* on ' 2026-02-19T20:16:31.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: service: mirror.2 on trial080 2026-02-19T20:16:31.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update 2026-02-19T20:16:31.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -nEe 's/^ last_update: *(.*) *$/\1/p' 2026-02-19T20:16:31.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:16:26' 2026-02-19T20:16:31.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying '2026-02-19 20:16:26' 2026-02-19T20:16:31.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:16:31.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:31.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:31.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state_pattern=up+replaying 2026-02-19T20:16:31.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local 'prev_last_update=2026-02-19 20:16:26' 2026-02-19T20:16:31.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status 2026-02-19T20:16:31.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ CEPH_ARGS= 2026-02-19T20:16:31.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2026-02-19T20:16:31.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status='OKOKOK142054290mirror.1trial08020.3.0-5336-g802bf111falseOK42084280mirror.2trial08020.3.0-5336-g802bf111trueOK44074410mirror.3trial08020.3.0-5336-g802bf111falseOK45044507mirror.0trial08020.3.0-5336-g802bf111falseOKtest845374c1-4dd1-4cdf-a0eb-5d7859e1dd45up+replayingreplaying, {"bytes_per_second":17878391.47,"bytes_per_snapshot":536568832.0,"last_snapshot_bytes":536351744,"last_snapshot_sync_seconds":0,"local_snapshot_timestamp":1771532177,"remote_snapshot_timestamp":1771532177,"replay_state":"idle"}42084280mirror.2trial0802026-02-19 20:16:26cluster2e7b2fa5d-8b41-4b57-afd6-1e94994e71e3up+stoppedlocal image is primary2026-02-19 20:16:26' 2026-02-19T20:16:31.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_update state 2026-02-19T20:16:31.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/last_update' 2026-02-19T20:16:31.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ last_update='2026-02-19 20:16:26' 2026-02-19T20:16:31.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v '//images/image[name='\''test'\'']/state' 2026-02-19T20:16:31.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=up+replaying 2026-02-19T20:16:31.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo up+replaying 2026-02-19T20:16:31.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep up+replaying 2026-02-19T20:16:31.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:up+replaying 2026-02-19T20:16:31.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:16:31.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:16:31.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap10 2026-02-19T20:16:31.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap10 2026-02-19T20:16:31.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:16:31.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:16:31.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:16:31.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap10 2026-02-19T20:16:31.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:16:54.371 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7775170566552299) > weight (0.25) 2026-02-19T20:16:54.371 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2026-02-19T20:16:54.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2026-02-19T20:16:54.372 INFO:tasks.rbd_mirror_thrash:waiting for 9.2098830882046 secs before reviving daemons 2026-02-19T20:16:54.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:16:54.370+0000 7f6e227aa640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 47831) UID: 1000 2026-02-19T20:17:00.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:17:00.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:00.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:00.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:00.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:17:00.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:00.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:00.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:00.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:17:00.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:17:00.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:17:00.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:17:01.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:17:01.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:17:01.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:17:01.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:17:01.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:17:01.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:17:01.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:17:01.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:17:01.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:17:01.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:17:01.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:17:01.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:17:01.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:17:01.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:17:01.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:17:01.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:17:01.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:17:01.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:17:01.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:17:01.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:17:01.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:17:01.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:17:01.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:17:01.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:17:01.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:17:01.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:17:01.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:17:01.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:17:01.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:17:01.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:17:01.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:17:01.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:17:01.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:17:01.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:17:01.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:17:01.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:17:01.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:17:01.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2026-02-19T20:17:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:17:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:17:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:17:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:17:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:17:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:17:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:17:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:17:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:17:01.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:17:01.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:17:01.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:17:01.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:17:01.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:17:01.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:17:01.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:17:01.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:17:01.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:17:01.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:17:01.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:17:01.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:17:01.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:01.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:01.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:01.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:17:03.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 32 2026-02-19T20:17:03.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:17:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:17:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:17:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:17:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:17:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:17:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:17:03.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:17:03.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:17:03.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:17:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:17:03.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:03.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:17:03.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:03.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:03.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:03.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:17:03.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:17:03.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:17:03.583 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2026-02-19T20:17:03.583 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:17:03.583 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2026-02-19T20:17:03.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2026-02-19T20:17:03.583 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2026-02-19T20:17:03.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2026-02-19T20:17:03.586 INFO:tasks.rbd_mirror_thrash:waiting for 62.38976916249905 secs before thrashing 2026-02-19T20:17:03.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:17:03.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=32 2026-02-19T20:17:03.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:17:03.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:17:03.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:17:03.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:03.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:03.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:03.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:03.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:03.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:17:03.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:17:03.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:17:03.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:17:03.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:17:03.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:17:03.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:17:03.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:17:03.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 32 = '' 2026-02-19T20:17:03.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:17:03.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:17:04.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:04.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:04.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:04.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:04.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:04.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:17:04.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:17:04.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:17:04.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:17:04.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:17:04.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:17:04.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:17:04.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:17:04.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 32 = '' 2026-02-19T20:17:04.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:17:04.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:17:05.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:05.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:05.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:05.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:05.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:05.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:17:05.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:17:05.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:17:05.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:17:05.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:17:05.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:17:05.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:17:05.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:17:05.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 32 = '' 2026-02-19T20:17:05.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:17:05.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:17:06.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:06.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:06.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:06.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:06.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:06.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:17:06.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:17:06.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:17:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=32 2026-02-19T20:17:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 32 = 32 2026-02-19T20:17:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap10 2026-02-19T20:17:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap10 2026-02-19T20:17:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:17:06.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:17:06.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:17:07.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap10 2026-02-19T20:17:07.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:17:07.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 20 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:17:07.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:15:33 2026 2026-02-19T20:17:07.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:17:07.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:17:07.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:17:07.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:17:07.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:17:07.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:17:07.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:07.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snap9-clean ']' 2026-02-19T20:17:07.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap9-clean 2026-02-19T20:17:07.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:07.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:07.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap9-clean 2026-02-19T20:17:07.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:07.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:07.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:07.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:07.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:08.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:08.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:08.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:08.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:08.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:08.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:08.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap10 2026-02-19T20:17:08.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:08.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:08.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap10 2026-02-19T20:17:08.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:08.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:08.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:08.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:08.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:09.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:09.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:09.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:09.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:10.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ clean_snap_name=snap10-clean 2026-02-19T20:17:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror test snap10-clean 2026-02-19T20:17:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:10.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:10.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap10-clean 2026-02-19T20:17:10.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10-clean --debug-rbd=20 --debug-journaler=20 2026-02-19T20:17:10.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror test 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:17:10.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:17:10.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:17:11.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror test started 2026-02-19T20:17:11.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:11.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:11.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:11.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/test 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:17:11.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:17:11.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:17:11.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:17:11.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:17:11.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/test 2026-02-19T20:17:11.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:11.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:17:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:17:11.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:17:11.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:17:11.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:17:11.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:17:11.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:17:11.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:17:11.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:17:11.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:17:11.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2026-02-19T20:17:11.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:17:11.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:17:11.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:17:11.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:17:11.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/test 2026-02-19T20:17:11.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:17:11.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:17:11.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:17:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:17:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:17:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:17:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:17:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:17:11.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:17:11.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:17:11.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:17:11.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2026-02-19T20:17:11.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/test'] 2026-02-19T20:17:11.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:17:11.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:17:11.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:17:11.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/test 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:17:11.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:17:11.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:17:11.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:17:11.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:17:11.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:17:11.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:17:11.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:17:11.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:17:11.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:17:11.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:17:11.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:17:11.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:17:11.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2026-02-19T20:17:11.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:17:11.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:17:11.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:17:11.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:17:11.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:17:11.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:17:11.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:17:11.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:11.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:17:11.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:17:11.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:11.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:17:11.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:17:11.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:11.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:17:11.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:17:11.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror test 2026-02-19T20:17:11.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:11.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:11.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:11.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2026-02-19T20:17:13.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 34 2026-02-19T20:17:13.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror test 2026-02-19T20:17:13.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:17:13.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:13.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:17:13.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:17:13.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:13.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-test.status 2026-02-19T20:17:13.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-test.status 2026-02-19T20:17:13.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:17:13.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:17:13.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-test.status 2026-02-19T20:17:13.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-test.status 2026-02-19T20:17:13.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:17:13.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:13.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:17:13.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:13.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:13.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:13.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.status 2026-02-19T20:17:13.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2026-02-19T20:17:13.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:17:13.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:17:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=34 2026-02-19T20:17:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:17:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:17:13.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:17:13.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:13.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:13.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:13.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:13.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:13.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:17:13.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:17:13.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:17:13.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:17:13.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:17:13.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:17:13.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:17:13.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:17:13.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 34 = '' 2026-02-19T20:17:13.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:17:13.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:17:14.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:14.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:14.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:14.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:14.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:14.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:17:14.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:17:14.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:17:14.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:17:14.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:17:14.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:17:14.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:17:14.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:17:14.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 34 = '' 2026-02-19T20:17:14.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:17:14.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:17:15.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:15.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:15.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:15.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:15.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.status 2026-02-19T20:17:15.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2026-02-19T20:17:15.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:17:15.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:17:15.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=34 2026-02-19T20:17:15.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 34 = 34 2026-02-19T20:17:15.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:15.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror test snap10-clean 2026-02-19T20:17:15.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:17:15.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:15.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:15.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap10-clean 2026-02-19T20:17:15.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:17:15.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:17:15.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:17:16.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/test@snap10-clean 2026-02-19T20:17:16.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'test': 2026-02-19T20:17:16.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:17:16.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:17:16.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 21 2026-02-19T20:17:16.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b811495b3e 2026-02-19T20:17:16.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b811495b3e 2026-02-19T20:17:16.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:17:16.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:17:16.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:17:16.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:17:16.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:17:16.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:17:08 2026 2026-02-19T20:17:16.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:08:58 2026 2026-02-19T20:17:16.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:17:16.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:17:16.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:17:16.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 845374c1-4dd1-4cdf-a0eb-5d7859e1dd45 2026-02-19T20:17:16.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 10 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap1 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap1 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap1 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:16.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:16.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:16.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:16.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:16.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:16.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:17.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:17.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:17.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap1-clean 2026-02-19T20:17:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap1-clean 2026-02-19T20:17:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap1-clean 2026-02-19T20:17:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:17.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:17.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:17.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:17.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:17.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:17.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:18.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:18.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:18.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:18.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:18.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:18.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap2 2026-02-19T20:17:18.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap2 2026-02-19T20:17:18.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:18.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:18.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap2 2026-02-19T20:17:18.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:18.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:18.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:18.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:18.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:18.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:18.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:19.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:19.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:19.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:19.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:19.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap2-clean 2026-02-19T20:17:19.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap2-clean 2026-02-19T20:17:19.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:19.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:19.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap2-clean 2026-02-19T20:17:19.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:19.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:19.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:19.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:19.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:20.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:20.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:20.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:20.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:20.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:21.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:21.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:21.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap3 2026-02-19T20:17:21.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap3 2026-02-19T20:17:21.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:21.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:21.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap3 2026-02-19T20:17:21.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:21.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:21.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:21.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:21.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:21.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:21.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:22.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:22.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:22.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:22.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap3-clean 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap3-clean 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap3-clean 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:22.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:22.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:23.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:23.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:23.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:23.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:23.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap4 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap4 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap4 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:23.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:23.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:23.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:24.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:24.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:24.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:24.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:24.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap4-clean 2026-02-19T20:17:24.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap4-clean 2026-02-19T20:17:24.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:24.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:24.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap4-clean 2026-02-19T20:17:24.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:24.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:24.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:24.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:24.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:25.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:25.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:25.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:25.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:25.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:25.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:25.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap5 2026-02-19T20:17:25.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap5 2026-02-19T20:17:25.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:25.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:25.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap5 2026-02-19T20:17:25.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:25.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:25.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:25.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:25.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:26.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:26.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:26.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:26.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:27.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:27.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:27.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap5-clean 2026-02-19T20:17:27.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap5-clean 2026-02-19T20:17:27.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:27.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:27.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap5-clean 2026-02-19T20:17:27.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:27.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:27.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:27.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:27.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:27.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:27.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:28.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:28.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:28.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:28.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap6 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap6 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap6 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:28.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:28.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:28.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:29.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:29.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:29.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap6-clean 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap6-clean 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap6-clean 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:29.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:30.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:30.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:30.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:30.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:30.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:30.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:30.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:30.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap7 2026-02-19T20:17:30.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap7 2026-02-19T20:17:30.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:30.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:30.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap7 2026-02-19T20:17:30.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:30.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:30.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:30.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:30.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:31.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:31.292 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:31.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:31.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:31.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap7-clean 2026-02-19T20:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap7-clean 2026-02-19T20:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap7-clean 2026-02-19T20:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:32.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:32.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:32.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:32.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:32.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:32.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:32.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:32.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:33.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:33.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:33.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:33.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap8 2026-02-19T20:17:33.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap8 2026-02-19T20:17:33.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:33.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:33.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap8 2026-02-19T20:17:33.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:33.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:33.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:33.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:33.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:33.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:33.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:34.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:34.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:34.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:34.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:34.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap8-clean 2026-02-19T20:17:34.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap8-clean 2026-02-19T20:17:34.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:34.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:34.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap8-clean 2026-02-19T20:17:34.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:34.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:34.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:34.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:34.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:34.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:34.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:35.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:35.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:35.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:35.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap9 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap9 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap9 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:35.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:36.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:36.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:36.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:36.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:36.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:36.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:36.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap9-clean 2026-02-19T20:17:36.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap9-clean 2026-02-19T20:17:36.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:36.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:36.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap9-clean 2026-02-19T20:17:36.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:36.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:36.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:36.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:36.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:37.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:37.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:37.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:37.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:38.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:38.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:38.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:38.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap10 2026-02-19T20:17:38.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap10 2026-02-19T20:17:38.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:38.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:38.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap10 2026-02-19T20:17:38.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:38.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:38.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:38.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:38.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:38.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:38.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:39.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:39.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:39.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:39.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:39.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap10-clean 2026-02-19T20:17:39.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror test snap10-clean 2026-02-19T20:17:39.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:39.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:39.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap10-clean 2026-02-19T20:17:39.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:17:39.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:39.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:39.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:39.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10-clean /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export 2026-02-19T20:17:39.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:39.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10-clean /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:40.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2026-02-19T20:17:40.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:40.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-test.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-test.export 2026-02-19T20:17:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:17:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 10 2026-02-19T20:17:40.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:40.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap1 2026-02-19T20:17:40.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap1 2026-02-19T20:17:40.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:40.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:40.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:40.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap1 2026-02-19T20:17:40.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1 2026-02-19T20:17:41.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:41.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap1-clean 2026-02-19T20:17:41.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap1-clean 2026-02-19T20:17:41.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:41.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:41.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:41.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap1-clean 2026-02-19T20:17:41.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1-clean 2026-02-19T20:17:42.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:42.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:42.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap2 2026-02-19T20:17:42.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap2 2026-02-19T20:17:42.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:42.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:42.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:42.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap2 2026-02-19T20:17:42.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2 2026-02-19T20:17:43.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:43.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap2-clean 2026-02-19T20:17:43.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap2-clean 2026-02-19T20:17:43.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:43.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:43.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:43.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap2-clean 2026-02-19T20:17:43.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2-clean 2026-02-19T20:17:44.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:44.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:44.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap3 2026-02-19T20:17:44.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap3 2026-02-19T20:17:44.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:44.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:44.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:44.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap3 2026-02-19T20:17:44.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3 2026-02-19T20:17:45.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:45.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap3-clean 2026-02-19T20:17:45.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap3-clean 2026-02-19T20:17:45.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:45.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:45.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:45.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap3-clean 2026-02-19T20:17:45.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3-clean 2026-02-19T20:17:46.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:46.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:46.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap4 2026-02-19T20:17:46.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap4 2026-02-19T20:17:46.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:46.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:46.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:46.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap4 2026-02-19T20:17:46.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4 2026-02-19T20:17:47.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:47.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap4-clean 2026-02-19T20:17:47.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap4-clean 2026-02-19T20:17:47.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:47.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:47.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:47.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap4-clean 2026-02-19T20:17:47.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4-clean 2026-02-19T20:17:48.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:48.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:48.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap5 2026-02-19T20:17:48.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap5 2026-02-19T20:17:48.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:48.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:48.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:48.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap5 2026-02-19T20:17:48.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5 2026-02-19T20:17:49.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:49.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap5-clean 2026-02-19T20:17:49.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap5-clean 2026-02-19T20:17:49.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:49.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:49.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:49.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap5-clean 2026-02-19T20:17:49.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5-clean 2026-02-19T20:17:50.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:50.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap6 2026-02-19T20:17:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap6 2026-02-19T20:17:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap6 2026-02-19T20:17:50.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6 2026-02-19T20:17:51.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:51.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap6-clean 2026-02-19T20:17:51.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap6-clean 2026-02-19T20:17:51.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:51.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:51.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:51.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap6-clean 2026-02-19T20:17:51.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6-clean 2026-02-19T20:17:52.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:52.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:52.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap7 2026-02-19T20:17:52.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap7 2026-02-19T20:17:52.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:52.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:52.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:52.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap7 2026-02-19T20:17:52.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7 2026-02-19T20:17:53.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:53.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap7-clean 2026-02-19T20:17:53.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap7-clean 2026-02-19T20:17:53.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:53.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:53.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:53.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap7-clean 2026-02-19T20:17:53.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7-clean 2026-02-19T20:17:54.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:54.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:54.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap8 2026-02-19T20:17:54.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap8 2026-02-19T20:17:54.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:54.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:54.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:54.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap8 2026-02-19T20:17:54.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8 2026-02-19T20:17:55.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:55.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap8-clean 2026-02-19T20:17:55.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap8-clean 2026-02-19T20:17:55.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:55.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:55.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:55.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap8-clean 2026-02-19T20:17:55.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8-clean 2026-02-19T20:17:56.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:56.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:56.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap9 2026-02-19T20:17:56.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap9 2026-02-19T20:17:56.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:56.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:56.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:56.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap9 2026-02-19T20:17:56.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9 2026-02-19T20:17:57.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:57.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap9-clean 2026-02-19T20:17:57.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap9-clean 2026-02-19T20:17:57.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:57.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:57.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:57.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap9-clean 2026-02-19T20:17:57.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9-clean 2026-02-19T20:17:58.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:58.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:17:58.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap10 2026-02-19T20:17:58.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap10 2026-02-19T20:17:58.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:58.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:58.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:58.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap10 2026-02-19T20:17:58.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10 2026-02-19T20:17:59.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:17:59.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap10-clean 2026-02-19T20:17:59.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror test snap10-clean 2026-02-19T20:17:59.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:17:59.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:17:59.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:17:59.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap10-clean 2026-02-19T20:17:59.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10-clean 2026-02-19T20:18:00.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:18:00.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror test 2026-02-19T20:18:00.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:00.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:00.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:18:00.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:18:00.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:18:00.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror test 2026-02-19T20:18:00.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:00.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:00.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:18:00.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/test 2026-02-19T20:18:00.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/test 2026-02-19T20:18:00.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:18:00.671+0000 7f9e07814640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a241a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:18:00.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:18:00.671+0000 7f9e0d820640 -1 rbd::mirror::ImageReplayer: 0x1b05680 [3/845374c1-4dd1-4cdf-a0eb-5d7859e1dd45] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:18:01.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:18:01.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:01.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror test deleted 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror test 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:18:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=test 2026-02-19T20:18:01.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/test 2026-02-19T20:18:01.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:18:01.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image test: (2) No such file or directory 2026-02-19T20:18:01.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:18:01.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:18:01.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:18:01.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror test deleted '' 2026-02-19T20:18:01.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=test 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror test 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=test 2026-02-19T20:18:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/test 2026-02-19T20:18:02.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:18:02.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image test: (2) No such file or directory 2026-02-19T20:18:02.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:18:02.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:18:02.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:18:02.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:02.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ testlog 'TEST: create many images' 2026-02-19T20:18:02.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ tee -a /tmp/tmp.9BL5XQhnTi/rbd-mirror.test.log 2026-02-19T20:18:02.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ date '+%F %T' 2026-02-19T20:18:02.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo 2026-02-19 20:18:02 TEST: create many images 2026-02-19T20:18:02.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:2026-02-19 20:18:02 TEST: create many images 2026-02-19T20:18:02.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ snap_name=snap 2026-02-19T20:18:02.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 50 2026-02-19T20:18:02.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:18:02.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_1 2026-02-19T20:18:02.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_1 snapshot 128M 2026-02-19T20:18:02.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:02.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:02.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_1 128M 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:18:02.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:02.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:18:02.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:18:02.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:18:02.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:02.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_1 2026-02-19T20:18:02.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:18:02.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:02.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:18:02.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:02.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_1 snapshot 2026-02-19T20:18:02.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:02.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:02.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:18:02.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:02.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_1 snapshot 2026-02-19T20:18:02.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:18:02.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_1 2026-02-19T20:18:02.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_1': 2026-02-19T20:18:02.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:18:02.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:18:02.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 126350472d9c 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.126350472d9c 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:02 2026 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:02 2026 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:02 2026 2026-02-19T20:18:02.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:18:02.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:18:02.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 1bb58447-6e84-49f6-86ad-1db8d6f29daf 2026-02-19T20:18:02.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:18:02.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:18:02.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_1 2026-02-19T20:18:02.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:02.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:02.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:18:02.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:18:02.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:18:02.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:18:02.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_1 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:18:02.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 21616640 bytes 2026-02-19T20:18:02.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 70709248 bytes 2026-02-19T20:18:02.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 122839040 bytes 2026-02-19T20:18:02.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 128162816 bytes 2026-02-19T20:18:02.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 176382976 bytes 2026-02-19T20:18:02.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 242825216 bytes 2026-02-19T20:18:03.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 268120064 bytes 2026-02-19T20:18:03.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 356699136 bytes 2026-02-19T20:18:03.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 435868672 bytes 2026-02-19T20:18:03.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 442347520 bytes 2026-02-19T20:18:03.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 476567552 bytes 2026-02-19T20:18:03.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 531841024 bytes 2026-02-19T20:18:03.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 567313408 bytes 2026-02-19T20:18:03.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 626285568 bytes 2026-02-19T20:18:03.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 660988928 bytes 2026-02-19T20:18:03.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 731220992 bytes 2026-02-19T20:18:03.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 779809792 bytes 2026-02-19T20:18:03.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 808188928 bytes 2026-02-19T20:18:03.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 815450112 bytes 2026-02-19T20:18:03.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 913833984 bytes 2026-02-19T20:18:03.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 958067712 bytes 2026-02-19T20:18:03.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1005770752 bytes 2026-02-19T20:18:03.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1036234752 bytes 2026-02-19T20:18:04.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1088434176 bytes 2026-02-19T20:18:04.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1198528512 bytes 2026-02-19T20:18:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1206626304 bytes 2026-02-19T20:18:04.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1251804160 bytes 2026-02-19T20:18:04.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1332431872 bytes 2026-02-19T20:18:04.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1370312704 bytes 2026-02-19T20:18:04.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1462227968 bytes 2026-02-19T20:18:04.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1496196096 bytes 2026-02-19T20:18:04.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1559194624 bytes 2026-02-19T20:18:04.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1577860096 bytes 2026-02-19T20:18:04.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1591693312 bytes 2026-02-19T20:18:04.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1613009920 bytes 2026-02-19T20:18:04.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1652157440 bytes 2026-02-19T20:18:04.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1684812800 bytes 2026-02-19T20:18:04.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1730522112 bytes 2026-02-19T20:18:04.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1784716288 bytes 2026-02-19T20:18:04.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1810729984 bytes 2026-02-19T20:18:05.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1926974464 bytes 2026-02-19T20:18:05.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 2008822784 bytes 2026-02-19T20:18:05.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2082709504 bytes 2026-02-19T20:18:05.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2104292352 bytes 2026-02-19T20:18:05.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2119120896 bytes 2026-02-19T20:18:05.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2133970944 bytes 2026-02-19T20:18:05.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2153586688 bytes 2026-02-19T20:18:05.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2232619008 bytes 2026-02-19T20:18:05.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2257264640 bytes 2026-02-19T20:18:05.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2298147840 bytes 2026-02-19T20:18:05.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2367328256 bytes 2026-02-19T20:18:05.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2407191552 bytes 2026-02-19T20:18:05.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2477918208 bytes 2026-02-19T20:18:05.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2535708672 bytes 2026-02-19T20:18:05.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2584118272 bytes 2026-02-19T20:18:05.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2596754432 bytes 2026-02-19T20:18:05.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2635144192 bytes 2026-02-19T20:18:05.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2665663488 bytes 2026-02-19T20:18:05.976 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2026-02-19T20:18:05.976 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2026-02-19T20:18:05.977 INFO:tasks.rbd_mirror_thrash:waiting for 4.918053159345673 secs before reviving daemons 2026-02-19T20:18:05.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:18:05.975+0000 7f14143b5640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 53486) UID: 1000 2026-02-19T20:18:06.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2714558464 bytes 2026-02-19T20:18:06.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2758624256 bytes 2026-02-19T20:18:06.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2844840960 bytes 2026-02-19T20:18:06.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2933913600 bytes 2026-02-19T20:18:06.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2934522880 bytes 2026-02-19T20:18:06.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2948313088 bytes 2026-02-19T20:18:06.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2991551488 bytes 2026-02-19T20:18:06.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3006937088 bytes 2026-02-19T20:18:06.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3062874112 bytes 2026-02-19T20:18:06.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3101123584 bytes 2026-02-19T20:18:06.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3123655680 bytes 2026-02-19T20:18:06.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3167703040 bytes 2026-02-19T20:18:06.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3200789504 bytes 2026-02-19T20:18:06.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3241340928 bytes 2026-02-19T20:18:06.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3266027520 bytes 2026-02-19T20:18:06.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3289924608 bytes 2026-02-19T20:18:06.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3312274432 bytes 2026-02-19T20:18:06.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3370092544 bytes 2026-02-19T20:18:06.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3438729216 bytes 2026-02-19T20:18:06.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3489570816 bytes 2026-02-19T20:18:07.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3522674688 bytes 2026-02-19T20:18:07.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3527460864 bytes 2026-02-19T20:18:07.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3567852544 bytes 2026-02-19T20:18:07.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3621289984 bytes 2026-02-19T20:18:07.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3644923904 bytes 2026-02-19T20:18:07.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3650161664 bytes 2026-02-19T20:18:07.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3714619392 bytes 2026-02-19T20:18:07.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3783654400 bytes 2026-02-19T20:18:07.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3824043008 bytes 2026-02-19T20:18:07.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3864289280 bytes 2026-02-19T20:18:07.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3895198720 bytes 2026-02-19T20:18:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3952564224 bytes 2026-02-19T20:18:07.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3957636096 bytes 2026-02-19T20:18:07.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4009338880 bytes 2026-02-19T20:18:07.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4064642048 bytes 2026-02-19T20:18:07.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4066365440 bytes 2026-02-19T20:18:07.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4136316928 bytes 2026-02-19T20:18:07.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4177931264 bytes 2026-02-19T20:18:07.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4197934080 bytes 2026-02-19T20:18:07.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4253090816 bytes 2026-02-19T20:18:08.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4293319680 bytes 2026-02-19T20:18:08.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4333548544 bytes 2026-02-19T20:18:08.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4424864768 bytes 2026-02-19T20:18:08.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4474611712 bytes 2026-02-19T20:18:08.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4493093888 bytes 2026-02-19T20:18:08.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4546081792 bytes 2026-02-19T20:18:08.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4610041856 bytes 2026-02-19T20:18:08.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4641906688 bytes 2026-02-19T20:18:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4686533632 bytes 2026-02-19T20:18:08.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4749528064 bytes 2026-02-19T20:18:08.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4757248000 bytes 2026-02-19T20:18:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4786556928 bytes 2026-02-19T20:18:08.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4824198144 bytes 2026-02-19T20:18:08.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4906644480 bytes 2026-02-19T20:18:08.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4968855552 bytes 2026-02-19T20:18:08.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5008716800 bytes 2026-02-19T20:18:08.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5050945536 bytes 2026-02-19T20:18:09.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5063318528 bytes 2026-02-19T20:18:09.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5110743040 bytes 2026-02-19T20:18:09.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5138892800 bytes 2026-02-19T20:18:09.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5139517440 bytes 2026-02-19T20:18:09.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5168774144 bytes 2026-02-19T20:18:09.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5272155136 bytes 2026-02-19T20:18:09.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5330396160 bytes 2026-02-19T20:18:09.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5361898496 bytes 2026-02-19T20:18:09.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5385922560 bytes 2026-02-19T20:18:09.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5420894208 bytes 2026-02-19T20:18:09.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5465219072 bytes 2026-02-19T20:18:09.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5500115968 bytes 2026-02-19T20:18:09.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5519985664 bytes 2026-02-19T20:18:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5538797568 bytes 2026-02-19T20:18:09.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5624881152 bytes 2026-02-19T20:18:09.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5628337152 bytes 2026-02-19T20:18:09.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5657660416 bytes 2026-02-19T20:18:09.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5662902272 bytes 2026-02-19T20:18:09.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5676324864 bytes 2026-02-19T20:18:09.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5693371392 bytes 2026-02-19T20:18:09.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5723074560 bytes 2026-02-19T20:18:10.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5832381440 bytes 2026-02-19T20:18:10.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5843947520 bytes 2026-02-19T20:18:10.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5847798784 bytes 2026-02-19T20:18:10.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5882853376 bytes 2026-02-19T20:18:10.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5907259392 bytes 2026-02-19T20:18:10.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5931348992 bytes 2026-02-19T20:18:10.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5969427456 bytes 2026-02-19T20:18:10.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5983935488 bytes 2026-02-19T20:18:10.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6018439168 bytes 2026-02-19T20:18:10.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6070605824 bytes 2026-02-19T20:18:10.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6138462208 bytes 2026-02-19T20:18:10.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6175247360 bytes 2026-02-19T20:18:10.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6234254336 bytes 2026-02-19T20:18:10.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6291002368 bytes 2026-02-19T20:18:10.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6328007680 bytes 2026-02-19T20:18:10.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6335589376 bytes 2026-02-19T20:18:10.896 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2026-02-19T20:18:10.896 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:18:10.896 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2026-02-19T20:18:10.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2026-02-19T20:18:10.896 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2026-02-19T20:18:10.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2026-02-19T20:18:10.900 INFO:tasks.rbd_mirror_thrash:waiting for 115.6293906182135 secs before thrashing 2026-02-19T20:18:10.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6410096640 bytes 2026-02-19T20:18:10.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6453447680 bytes 2026-02-19T20:18:11.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6475199488 bytes 2026-02-19T20:18:11.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6475830272 bytes 2026-02-19T20:18:11.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6530981888 bytes 2026-02-19T20:18:11.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6603319296 bytes 2026-02-19T20:18:11.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6635400192 bytes 2026-02-19T20:18:11.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6681841664 bytes 2026-02-19T20:18:11.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6772909056 bytes 2026-02-19T20:18:11.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6773517312 bytes 2026-02-19T20:18:11.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6850803712 bytes 2026-02-19T20:18:11.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6872733696 bytes 2026-02-19T20:18:11.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6903805952 bytes 2026-02-19T20:18:11.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6904818688 bytes 2026-02-19T20:18:11.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6938318848 bytes 2026-02-19T20:18:11.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6995389440 bytes 2026-02-19T20:18:11.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7068264448 bytes 2026-02-19T20:18:11.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7078220800 bytes 2026-02-19T20:18:11.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7108098048 bytes 2026-02-19T20:18:11.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7125240832 bytes 2026-02-19T20:18:11.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7146935296 bytes 2026-02-19T20:18:11.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7180689408 bytes 2026-02-19T20:18:12.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7221371904 bytes 2026-02-19T20:18:12.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7278304256 bytes 2026-02-19T20:18:12.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7336991744 bytes 2026-02-19T20:18:12.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7374771200 bytes 2026-02-19T20:18:12.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7387236352 bytes 2026-02-19T20:18:12.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7453736960 bytes 2026-02-19T20:18:12.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7461266432 bytes 2026-02-19T20:18:12.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7483279360 bytes 2026-02-19T20:18:12.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7524592640 bytes 2026-02-19T20:18:12.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7547525120 bytes 2026-02-19T20:18:12.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7563632640 bytes 2026-02-19T20:18:12.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7621822464 bytes 2026-02-19T20:18:12.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7684797440 bytes 2026-02-19T20:18:12.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7703075840 bytes 2026-02-19T20:18:12.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7750566912 bytes 2026-02-19T20:18:12.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7797791744 bytes 2026-02-19T20:18:12.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7822914560 bytes 2026-02-19T20:18:12.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7854420992 bytes 2026-02-19T20:18:13.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7904109568 bytes 2026-02-19T20:18:13.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 7951712256 bytes 2026-02-19T20:18:13.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 7974083584 bytes 2026-02-19T20:18:13.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8079477760 bytes 2026-02-19T20:18:13.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8093646848 bytes 2026-02-19T20:18:13.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8096656384 bytes 2026-02-19T20:18:13.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8123794432 bytes 2026-02-19T20:18:13.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8208090112 bytes 2026-02-19T20:18:13.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8286533632 bytes 2026-02-19T20:18:13.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8318939136 bytes 2026-02-19T20:18:13.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8358800384 bytes 2026-02-19T20:18:13.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8383602688 bytes 2026-02-19T20:18:13.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8428797952 bytes 2026-02-19T20:18:13.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8496109568 bytes 2026-02-19T20:18:13.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8553672704 bytes 2026-02-19T20:18:13.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8602895360 bytes 2026-02-19T20:18:13.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 8620656640 bytes 2026-02-19T20:18:14.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 8688527360 bytes 2026-02-19T20:18:14.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 8701680640 bytes 2026-02-19T20:18:14.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 8780357632 bytes 2026-02-19T20:18:14.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 8822318080 bytes 2026-02-19T20:18:14.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 8872516608 bytes 2026-02-19T20:18:14.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 8915662848 bytes 2026-02-19T20:18:14.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 8961448960 bytes 2026-02-19T20:18:14.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 8987612160 bytes 2026-02-19T20:18:14.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 8991690752 bytes 2026-02-19T20:18:14.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 9012026368 bytes 2026-02-19T20:18:14.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 9031181312 bytes 2026-02-19T20:18:14.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 9056529408 bytes 2026-02-19T20:18:14.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 9108532224 bytes 2026-02-19T20:18:14.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 9149824000 bytes 2026-02-19T20:18:14.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 9189796864 bytes 2026-02-19T20:18:14.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 9255521280 bytes 2026-02-19T20:18:14.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22600 IOs, 9308589056 bytes 2026-02-19T20:18:14.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22700 IOs, 9366789120 bytes 2026-02-19T20:18:15.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22800 IOs, 9375770624 bytes 2026-02-19T20:18:15.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22900 IOs, 9429723136 bytes 2026-02-19T20:18:15.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23000 IOs, 9492973568 bytes 2026-02-19T20:18:15.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23100 IOs, 9493593088 bytes 2026-02-19T20:18:15.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23200 IOs, 9511363584 bytes 2026-02-19T20:18:15.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23300 IOs, 9525511168 bytes 2026-02-19T20:18:15.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23400 IOs, 9547372544 bytes 2026-02-19T20:18:15.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23500 IOs, 9595153408 bytes 2026-02-19T20:18:15.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23600 IOs, 9668036608 bytes 2026-02-19T20:18:15.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23700 IOs, 9713472512 bytes 2026-02-19T20:18:15.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23800 IOs, 9747076096 bytes 2026-02-19T20:18:15.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23900 IOs, 9845587968 bytes 2026-02-19T20:18:15.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:18:15.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:18:15.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:18:15.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:15.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:18:15.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_2 2026-02-19T20:18:15.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_2 snapshot 128M 2026-02-19T20:18:15.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_2 128M 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:15.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:15.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:18:15.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:15.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:18:15.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:18:15.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:18:15.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:15.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_2 2026-02-19T20:18:15.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:18:15.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:15.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:18:15.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:15.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_2 snapshot 2026-02-19T20:18:15.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:15.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:15.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:18:15.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:15.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_2 snapshot 2026-02-19T20:18:16.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:18:16.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_2 2026-02-19T20:18:16.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_2': 2026-02-19T20:18:16.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:18:16.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1275565b548d 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1275565b548d 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:15 2026 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:15 2026 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:15 2026 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:18:16.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 26fa2d67-71a3-4278-b257-25f69a27f0fe 2026-02-19T20:18:16.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:18:16.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:18:16.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_2 2026-02-19T20:18:16.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:16.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:16.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:18:16.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:18:16.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:18:16.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:18:16.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_2 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:18:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 75787264 bytes 2026-02-19T20:18:16.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 150501376 bytes 2026-02-19T20:18:16.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 203280384 bytes 2026-02-19T20:18:16.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 226121728 bytes 2026-02-19T20:18:16.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 342405120 bytes 2026-02-19T20:18:17.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 403681280 bytes 2026-02-19T20:18:17.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 447193088 bytes 2026-02-19T20:18:17.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 472613888 bytes 2026-02-19T20:18:17.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 487852032 bytes 2026-02-19T20:18:17.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 527841280 bytes 2026-02-19T20:18:17.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 529215488 bytes 2026-02-19T20:18:17.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 563198976 bytes 2026-02-19T20:18:17.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 618829824 bytes 2026-02-19T20:18:17.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 641061888 bytes 2026-02-19T20:18:17.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 720463872 bytes 2026-02-19T20:18:17.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 727662592 bytes 2026-02-19T20:18:17.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 794400768 bytes 2026-02-19T20:18:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 837894144 bytes 2026-02-19T20:18:17.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 885541888 bytes 2026-02-19T20:18:17.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 969845760 bytes 2026-02-19T20:18:17.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1015771136 bytes 2026-02-19T20:18:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1035039744 bytes 2026-02-19T20:18:18.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1047570432 bytes 2026-02-19T20:18:18.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1070356480 bytes 2026-02-19T20:18:18.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1125524480 bytes 2026-02-19T20:18:18.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1163957248 bytes 2026-02-19T20:18:18.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1193208832 bytes 2026-02-19T20:18:18.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1242498048 bytes 2026-02-19T20:18:18.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1305126912 bytes 2026-02-19T20:18:18.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1388777472 bytes 2026-02-19T20:18:18.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1391179776 bytes 2026-02-19T20:18:18.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1429699584 bytes 2026-02-19T20:18:18.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1477068800 bytes 2026-02-19T20:18:18.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1511868416 bytes 2026-02-19T20:18:18.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1530381312 bytes 2026-02-19T20:18:18.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1558133760 bytes 2026-02-19T20:18:18.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1593467904 bytes 2026-02-19T20:18:18.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1637133312 bytes 2026-02-19T20:18:18.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1701751808 bytes 2026-02-19T20:18:19.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1747873792 bytes 2026-02-19T20:18:19.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1807408128 bytes 2026-02-19T20:18:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1809936384 bytes 2026-02-19T20:18:19.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1846539264 bytes 2026-02-19T20:18:19.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1946764288 bytes 2026-02-19T20:18:19.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1989484544 bytes 2026-02-19T20:18:19.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2017440768 bytes 2026-02-19T20:18:19.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2088358912 bytes 2026-02-19T20:18:19.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2111379456 bytes 2026-02-19T20:18:19.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2180171776 bytes 2026-02-19T20:18:19.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2220678144 bytes 2026-02-19T20:18:19.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2241728512 bytes 2026-02-19T20:18:19.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2264463360 bytes 2026-02-19T20:18:19.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2313856000 bytes 2026-02-19T20:18:19.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2338598912 bytes 2026-02-19T20:18:19.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2408683520 bytes 2026-02-19T20:18:20.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2411013120 bytes 2026-02-19T20:18:20.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2481902592 bytes 2026-02-19T20:18:20.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2510269440 bytes 2026-02-19T20:18:20.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2532192256 bytes 2026-02-19T20:18:20.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2600304640 bytes 2026-02-19T20:18:20.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2685293568 bytes 2026-02-19T20:18:20.439 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2765769728 bytes 2026-02-19T20:18:20.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2823334912 bytes 2026-02-19T20:18:20.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2880046080 bytes 2026-02-19T20:18:20.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2918561792 bytes 2026-02-19T20:18:20.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2935887872 bytes 2026-02-19T20:18:20.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2975787008 bytes 2026-02-19T20:18:20.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3006559232 bytes 2026-02-19T20:18:20.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3087809536 bytes 2026-02-19T20:18:20.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3088402432 bytes 2026-02-19T20:18:21.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3168334848 bytes 2026-02-19T20:18:21.077 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3196141568 bytes 2026-02-19T20:18:21.103 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3196761088 bytes 2026-02-19T20:18:21.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3237812224 bytes 2026-02-19T20:18:21.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3285819392 bytes 2026-02-19T20:18:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3364456448 bytes 2026-02-19T20:18:21.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3365073920 bytes 2026-02-19T20:18:21.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3390776320 bytes 2026-02-19T20:18:21.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3422066688 bytes 2026-02-19T20:18:21.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3422687232 bytes 2026-02-19T20:18:21.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3487068160 bytes 2026-02-19T20:18:21.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3528942592 bytes 2026-02-19T20:18:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3582960640 bytes 2026-02-19T20:18:21.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3617421312 bytes 2026-02-19T20:18:21.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3634190336 bytes 2026-02-19T20:18:21.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3724990464 bytes 2026-02-19T20:18:21.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3783317504 bytes 2026-02-19T20:18:22.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3799183360 bytes 2026-02-19T20:18:22.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3822547968 bytes 2026-02-19T20:18:22.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3823172608 bytes 2026-02-19T20:18:22.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3919279104 bytes 2026-02-19T20:18:22.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4006349824 bytes 2026-02-19T20:18:22.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4066819072 bytes 2026-02-19T20:18:22.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4071889920 bytes 2026-02-19T20:18:22.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4170316800 bytes 2026-02-19T20:18:22.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4208282624 bytes 2026-02-19T20:18:22.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4219040768 bytes 2026-02-19T20:18:22.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4260695040 bytes 2026-02-19T20:18:22.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4306565120 bytes 2026-02-19T20:18:22.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4363574272 bytes 2026-02-19T20:18:22.821 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7536220715041916) > weight (0.25) 2026-02-19T20:18:22.821 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9196732048191122) > weight (0.25) 2026-02-19T20:18:22.821 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9099993817595757) > weight (0.25) 2026-02-19T20:18:22.821 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4438910268822791) > weight (0.25) 2026-02-19T20:18:22.821 INFO:tasks.rbd_mirror_thrash:waiting for 101.65189420944762 secs before thrashing 2026-02-19T20:18:22.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4385611776 bytes 2026-02-19T20:18:22.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4403205120 bytes 2026-02-19T20:18:22.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4443581440 bytes 2026-02-19T20:18:23.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4502832128 bytes 2026-02-19T20:18:23.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4572746752 bytes 2026-02-19T20:18:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4610104320 bytes 2026-02-19T20:18:23.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4613008384 bytes 2026-02-19T20:18:23.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4657822720 bytes 2026-02-19T20:18:23.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4714470400 bytes 2026-02-19T20:18:23.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4800281600 bytes 2026-02-19T20:18:23.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4800884736 bytes 2026-02-19T20:18:23.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4867043328 bytes 2026-02-19T20:18:23.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4912311296 bytes 2026-02-19T20:18:23.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4960312320 bytes 2026-02-19T20:18:23.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4972686336 bytes 2026-02-19T20:18:23.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5014079488 bytes 2026-02-19T20:18:23.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5062671360 bytes 2026-02-19T20:18:23.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5164365824 bytes 2026-02-19T20:18:23.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5197725696 bytes 2026-02-19T20:18:24.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5276568576 bytes 2026-02-19T20:18:24.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5302388736 bytes 2026-02-19T20:18:24.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5361636352 bytes 2026-02-19T20:18:24.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5384316928 bytes 2026-02-19T20:18:24.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5398919168 bytes 2026-02-19T20:18:24.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5467128832 bytes 2026-02-19T20:18:24.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5536000000 bytes 2026-02-19T20:18:24.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5564632064 bytes 2026-02-19T20:18:24.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5597276160 bytes 2026-02-19T20:18:24.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5631068160 bytes 2026-02-19T20:18:24.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5680633856 bytes 2026-02-19T20:18:24.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5697004544 bytes 2026-02-19T20:18:24.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5725340672 bytes 2026-02-19T20:18:24.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5735878656 bytes 2026-02-19T20:18:24.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5806312448 bytes 2026-02-19T20:18:24.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5829509120 bytes 2026-02-19T20:18:24.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5872016384 bytes 2026-02-19T20:18:25.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5893240832 bytes 2026-02-19T20:18:25.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5934189568 bytes 2026-02-19T20:18:25.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5987867648 bytes 2026-02-19T20:18:25.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6017437696 bytes 2026-02-19T20:18:25.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6084800512 bytes 2026-02-19T20:18:25.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6090971136 bytes 2026-02-19T20:18:25.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6119402496 bytes 2026-02-19T20:18:25.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6181404672 bytes 2026-02-19T20:18:25.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6227912704 bytes 2026-02-19T20:18:25.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6270408704 bytes 2026-02-19T20:18:25.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6344787968 bytes 2026-02-19T20:18:25.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6439704576 bytes 2026-02-19T20:18:25.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6450875392 bytes 2026-02-19T20:18:25.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6479967232 bytes 2026-02-19T20:18:25.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6509225984 bytes 2026-02-19T20:18:25.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6540572672 bytes 2026-02-19T20:18:26.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6584330240 bytes 2026-02-19T20:18:26.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6645862400 bytes 2026-02-19T20:18:26.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6698213376 bytes 2026-02-19T20:18:26.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6702985216 bytes 2026-02-19T20:18:26.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6718543872 bytes 2026-02-19T20:18:26.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6783601664 bytes 2026-02-19T20:18:26.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6835702784 bytes 2026-02-19T20:18:26.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6896860160 bytes 2026-02-19T20:18:26.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6911247360 bytes 2026-02-19T20:18:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_3 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_3 snapshot 128M 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_3 128M 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:18:26.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:26.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:18:26.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:18:26.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:18:26.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:26.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_3 2026-02-19T20:18:26.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:18:26.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:26.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:18:26.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:26.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_3 snapshot 2026-02-19T20:18:26.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:26.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:26.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:18:26.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:26.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_3 snapshot 2026-02-19T20:18:27.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:18:27.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_3 2026-02-19T20:18:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_3': 2026-02-19T20:18:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:18:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:18:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:18:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1281a0adaf82 2026-02-19T20:18:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1281a0adaf82 2026-02-19T20:18:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:18:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:18:27.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:18:27.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:18:27.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:26 2026 2026-02-19T20:18:27.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:26 2026 2026-02-19T20:18:27.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:26 2026 2026-02-19T20:18:27.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:18:27.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:18:27.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 71651990-b478-4ee5-835f-7147f7defb72 2026-02-19T20:18:27.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:18:27.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:18:27.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_3 2026-02-19T20:18:27.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:27.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:27.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:18:27.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:18:27.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:18:27.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:18:27.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_3 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:18:27.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 29620224 bytes 2026-02-19T20:18:27.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 81895424 bytes 2026-02-19T20:18:27.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 150049792 bytes 2026-02-19T20:18:27.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 150656000 bytes 2026-02-19T20:18:27.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 206736384 bytes 2026-02-19T20:18:27.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 222244864 bytes 2026-02-19T20:18:27.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 225649664 bytes 2026-02-19T20:18:28.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 315431936 bytes 2026-02-19T20:18:28.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 343847936 bytes 2026-02-19T20:18:28.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 382858240 bytes 2026-02-19T20:18:28.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 413259776 bytes 2026-02-19T20:18:28.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 454202368 bytes 2026-02-19T20:18:28.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 487863296 bytes 2026-02-19T20:18:28.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 525110272 bytes 2026-02-19T20:18:28.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 542699520 bytes 2026-02-19T20:18:28.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 581432320 bytes 2026-02-19T20:18:28.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 626378752 bytes 2026-02-19T20:18:28.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 718978048 bytes 2026-02-19T20:18:28.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 769570816 bytes 2026-02-19T20:18:28.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 855217152 bytes 2026-02-19T20:18:28.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 902121472 bytes 2026-02-19T20:18:28.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 912409600 bytes 2026-02-19T20:18:28.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 934633472 bytes 2026-02-19T20:18:28.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 972248064 bytes 2026-02-19T20:18:28.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1005252608 bytes 2026-02-19T20:18:28.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1040538624 bytes 2026-02-19T20:18:29.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1111135232 bytes 2026-02-19T20:18:29.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1165787136 bytes 2026-02-19T20:18:29.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1211345920 bytes 2026-02-19T20:18:29.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1240667136 bytes 2026-02-19T20:18:29.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1298483200 bytes 2026-02-19T20:18:29.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1353065472 bytes 2026-02-19T20:18:29.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1443341312 bytes 2026-02-19T20:18:29.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1468934144 bytes 2026-02-19T20:18:29.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1547891712 bytes 2026-02-19T20:18:29.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1637548032 bytes 2026-02-19T20:18:29.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1669031936 bytes 2026-02-19T20:18:29.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1679980544 bytes 2026-02-19T20:18:29.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1762997248 bytes 2026-02-19T20:18:29.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1792060416 bytes 2026-02-19T20:18:29.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1843273728 bytes 2026-02-19T20:18:29.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1902982144 bytes 2026-02-19T20:18:30.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1972640768 bytes 2026-02-19T20:18:30.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2082442240 bytes 2026-02-19T20:18:30.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2126290944 bytes 2026-02-19T20:18:30.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2180922368 bytes 2026-02-19T20:18:30.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2221830144 bytes 2026-02-19T20:18:30.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2283212800 bytes 2026-02-19T20:18:30.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2335185920 bytes 2026-02-19T20:18:30.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2343449600 bytes 2026-02-19T20:18:30.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2391111680 bytes 2026-02-19T20:18:30.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2456741888 bytes 2026-02-19T20:18:30.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2457353216 bytes 2026-02-19T20:18:30.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2501879808 bytes 2026-02-19T20:18:30.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2522231808 bytes 2026-02-19T20:18:30.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2579591168 bytes 2026-02-19T20:18:30.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2580224000 bytes 2026-02-19T20:18:30.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2620596224 bytes 2026-02-19T20:18:30.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2642779136 bytes 2026-02-19T20:18:31.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2722946048 bytes 2026-02-19T20:18:31.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2780646400 bytes 2026-02-19T20:18:31.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2885919744 bytes 2026-02-19T20:18:31.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2946604032 bytes 2026-02-19T20:18:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2992775168 bytes 2026-02-19T20:18:31.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3012393984 bytes 2026-02-19T20:18:31.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3086016512 bytes 2026-02-19T20:18:31.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3146043392 bytes 2026-02-19T20:18:31.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3168834560 bytes 2026-02-19T20:18:31.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3223989248 bytes 2026-02-19T20:18:31.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3311766528 bytes 2026-02-19T20:18:31.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3378922496 bytes 2026-02-19T20:18:31.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3455590400 bytes 2026-02-19T20:18:31.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3522404352 bytes 2026-02-19T20:18:31.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3556849664 bytes 2026-02-19T20:18:32.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3622695936 bytes 2026-02-19T20:18:32.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3650236416 bytes 2026-02-19T20:18:32.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3689026560 bytes 2026-02-19T20:18:32.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3730262016 bytes 2026-02-19T20:18:32.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3771677696 bytes 2026-02-19T20:18:32.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3784102912 bytes 2026-02-19T20:18:32.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3845856256 bytes 2026-02-19T20:18:32.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3910340608 bytes 2026-02-19T20:18:32.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3963414528 bytes 2026-02-19T20:18:32.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 4035742720 bytes 2026-02-19T20:18:32.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 4067720192 bytes 2026-02-19T20:18:32.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 4104294400 bytes 2026-02-19T20:18:32.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4151883776 bytes 2026-02-19T20:18:32.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4218308608 bytes 2026-02-19T20:18:32.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4294385664 bytes 2026-02-19T20:18:32.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4331072512 bytes 2026-02-19T20:18:32.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4371152896 bytes 2026-02-19T20:18:33.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4388034560 bytes 2026-02-19T20:18:33.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4403427328 bytes 2026-02-19T20:18:33.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4473665536 bytes 2026-02-19T20:18:33.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4483595264 bytes 2026-02-19T20:18:33.230 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4533325824 bytes 2026-02-19T20:18:33.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4565726208 bytes 2026-02-19T20:18:33.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4579287040 bytes 2026-02-19T20:18:33.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4580055040 bytes 2026-02-19T20:18:33.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4663706624 bytes 2026-02-19T20:18:33.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4702227456 bytes 2026-02-19T20:18:33.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4727888896 bytes 2026-02-19T20:18:33.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4783175680 bytes 2026-02-19T20:18:33.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4873120768 bytes 2026-02-19T20:18:33.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4923681792 bytes 2026-02-19T20:18:33.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4950220800 bytes 2026-02-19T20:18:33.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4978245632 bytes 2026-02-19T20:18:33.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 5001953280 bytes 2026-02-19T20:18:33.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 5052741632 bytes 2026-02-19T20:18:34.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5089824768 bytes 2026-02-19T20:18:34.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5114820608 bytes 2026-02-19T20:18:34.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5150887936 bytes 2026-02-19T20:18:34.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5177630720 bytes 2026-02-19T20:18:34.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5220872192 bytes 2026-02-19T20:18:34.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5231995904 bytes 2026-02-19T20:18:34.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5274166272 bytes 2026-02-19T20:18:34.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5326229504 bytes 2026-02-19T20:18:34.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5413253120 bytes 2026-02-19T20:18:34.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5493263360 bytes 2026-02-19T20:18:34.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5577513984 bytes 2026-02-19T20:18:34.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5599227904 bytes 2026-02-19T20:18:34.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5664099328 bytes 2026-02-19T20:18:34.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5688984576 bytes 2026-02-19T20:18:34.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5719252992 bytes 2026-02-19T20:18:34.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5746107392 bytes 2026-02-19T20:18:34.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5829983232 bytes 2026-02-19T20:18:35.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5871324160 bytes 2026-02-19T20:18:35.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5925723136 bytes 2026-02-19T20:18:35.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5985490944 bytes 2026-02-19T20:18:35.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 6047495168 bytes 2026-02-19T20:18:35.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 6066493440 bytes 2026-02-19T20:18:35.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 6154170368 bytes 2026-02-19T20:18:35.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 6164793344 bytes 2026-02-19T20:18:35.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6165413888 bytes 2026-02-19T20:18:35.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6197796864 bytes 2026-02-19T20:18:35.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6244983808 bytes 2026-02-19T20:18:35.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6270060544 bytes 2026-02-19T20:18:35.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6305723392 bytes 2026-02-19T20:18:35.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6343568384 bytes 2026-02-19T20:18:35.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6344214528 bytes 2026-02-19T20:18:35.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6405998592 bytes 2026-02-19T20:18:35.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6411168768 bytes 2026-02-19T20:18:35.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6439022592 bytes 2026-02-19T20:18:35.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6488129536 bytes 2026-02-19T20:18:35.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6498494464 bytes 2026-02-19T20:18:35.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6504817664 bytes 2026-02-19T20:18:36.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6532652032 bytes 2026-02-19T20:18:36.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6578854912 bytes 2026-02-19T20:18:36.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6688071680 bytes 2026-02-19T20:18:36.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6737635328 bytes 2026-02-19T20:18:36.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6750001152 bytes 2026-02-19T20:18:36.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6776842240 bytes 2026-02-19T20:18:36.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6814184448 bytes 2026-02-19T20:18:36.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6844633088 bytes 2026-02-19T20:18:36.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6895993856 bytes 2026-02-19T20:18:36.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6936086528 bytes 2026-02-19T20:18:36.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6952417280 bytes 2026-02-19T20:18:36.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7012877312 bytes 2026-02-19T20:18:36.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7056015360 bytes 2026-02-19T20:18:37.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7095407616 bytes 2026-02-19T20:18:37.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7101153280 bytes 2026-02-19T20:18:37.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7158402048 bytes 2026-02-19T20:18:37.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7165650944 bytes 2026-02-19T20:18:37.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7178220544 bytes 2026-02-19T20:18:37.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7182852096 bytes 2026-02-19T20:18:37.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7204281344 bytes 2026-02-19T20:18:37.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7265337344 bytes 2026-02-19T20:18:37.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7325087744 bytes 2026-02-19T20:18:37.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7327015936 bytes 2026-02-19T20:18:37.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7380779008 bytes 2026-02-19T20:18:37.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7398020096 bytes 2026-02-19T20:18:37.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7415044096 bytes 2026-02-19T20:18:37.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7463325696 bytes 2026-02-19T20:18:37.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7497786368 bytes 2026-02-19T20:18:37.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7534385152 bytes 2026-02-19T20:18:37.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7597297664 bytes 2026-02-19T20:18:37.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7609235456 bytes 2026-02-19T20:18:37.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7656292352 bytes 2026-02-19T20:18:37.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7742041088 bytes 2026-02-19T20:18:38.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7823930368 bytes 2026-02-19T20:18:38.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7838551040 bytes 2026-02-19T20:18:38.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7847992320 bytes 2026-02-19T20:18:38.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7884120064 bytes 2026-02-19T20:18:38.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7931168768 bytes 2026-02-19T20:18:38.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7943096320 bytes 2026-02-19T20:18:38.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7951896576 bytes 2026-02-19T20:18:38.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8092436480 bytes 2026-02-19T20:18:38.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8166096896 bytes 2026-02-19T20:18:38.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8234020864 bytes 2026-02-19T20:18:38.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8257224704 bytes 2026-02-19T20:18:38.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8300953600 bytes 2026-02-19T20:18:38.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8339629056 bytes 2026-02-19T20:18:38.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8380393472 bytes 2026-02-19T20:18:38.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8425916416 bytes 2026-02-19T20:18:38.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8475181056 bytes 2026-02-19T20:18:38.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8525964288 bytes 2026-02-19T20:18:39.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8553698304 bytes 2026-02-19T20:18:39.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8611389440 bytes 2026-02-19T20:18:39.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8670495744 bytes 2026-02-19T20:18:39.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8710290432 bytes 2026-02-19T20:18:39.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8743012352 bytes 2026-02-19T20:18:39.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8785746944 bytes 2026-02-19T20:18:39.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8826320896 bytes 2026-02-19T20:18:39.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8826967040 bytes 2026-02-19T20:18:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8868776960 bytes 2026-02-19T20:18:39.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8886217728 bytes 2026-02-19T20:18:39.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8939552768 bytes 2026-02-19T20:18:39.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8961790976 bytes 2026-02-19T20:18:39.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 9025339392 bytes 2026-02-19T20:18:39.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 9025956864 bytes 2026-02-19T20:18:39.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 9104834560 bytes 2026-02-19T20:18:39.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 9121893376 bytes 2026-02-19T20:18:39.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 9142588416 bytes 2026-02-19T20:18:39.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 9158157312 bytes 2026-02-19T20:18:39.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 9191472128 bytes 2026-02-19T20:18:40.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 9210139648 bytes 2026-02-19T20:18:40.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 9210755072 bytes 2026-02-19T20:18:40.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 9273292800 bytes 2026-02-19T20:18:40.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 9315420160 bytes 2026-02-19T20:18:40.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 9338146816 bytes 2026-02-19T20:18:40.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 9364754432 bytes 2026-02-19T20:18:40.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 9382044672 bytes 2026-02-19T20:18:40.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 9406830592 bytes 2026-02-19T20:18:40.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 9466587136 bytes 2026-02-19T20:18:40.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 9487326208 bytes 2026-02-19T20:18:40.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22600 IOs, 9551697920 bytes 2026-02-19T20:18:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_4 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_4 snapshot 128M 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_4 128M 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:18:40.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:18:40.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:18:40.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:40.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_4 2026-02-19T20:18:40.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:18:40.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:40.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:18:40.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:40.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_4 snapshot 2026-02-19T20:18:40.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:18:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_4 snapshot 2026-02-19T20:18:41.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:18:41.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_4 2026-02-19T20:18:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_4': 2026-02-19T20:18:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:18:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:18:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:18:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 128dcb2d0bf0 2026-02-19T20:18:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.128dcb2d0bf0 2026-02-19T20:18:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:18:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:18:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:18:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:18:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:40 2026 2026-02-19T20:18:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:40 2026 2026-02-19T20:18:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:40 2026 2026-02-19T20:18:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:18:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:18:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 03107192-aafc-4953-a9c6-561e8976e03f 2026-02-19T20:18:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:18:41.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:18:41.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_4 2026-02-19T20:18:41.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:41.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:41.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:18:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:18:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:18:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:18:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_4 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:18:41.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 40204288 bytes 2026-02-19T20:18:41.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 72355840 bytes 2026-02-19T20:18:41.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 137098240 bytes 2026-02-19T20:18:41.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 193215488 bytes 2026-02-19T20:18:41.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 218101760 bytes 2026-02-19T20:18:41.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 262018048 bytes 2026-02-19T20:18:42.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 354378752 bytes 2026-02-19T20:18:42.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 394650624 bytes 2026-02-19T20:18:42.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 450222080 bytes 2026-02-19T20:18:42.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 475657216 bytes 2026-02-19T20:18:42.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 537630720 bytes 2026-02-19T20:18:42.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 538247168 bytes 2026-02-19T20:18:42.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 593039360 bytes 2026-02-19T20:18:42.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 600956928 bytes 2026-02-19T20:18:42.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 734609408 bytes 2026-02-19T20:18:42.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 735248384 bytes 2026-02-19T20:18:42.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 766936064 bytes 2026-02-19T20:18:42.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 768210944 bytes 2026-02-19T20:18:42.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 897558528 bytes 2026-02-19T20:18:42.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 921839616 bytes 2026-02-19T20:18:42.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 981088256 bytes 2026-02-19T20:18:42.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1050424320 bytes 2026-02-19T20:18:42.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1058927616 bytes 2026-02-19T20:18:43.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1096274944 bytes 2026-02-19T20:18:43.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1098278912 bytes 2026-02-19T20:18:43.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1098897408 bytes 2026-02-19T20:18:43.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1122140160 bytes 2026-02-19T20:18:43.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1176867840 bytes 2026-02-19T20:18:43.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1246922752 bytes 2026-02-19T20:18:43.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1315280896 bytes 2026-02-19T20:18:43.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1368207360 bytes 2026-02-19T20:18:43.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1423071232 bytes 2026-02-19T20:18:43.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1478142976 bytes 2026-02-19T20:18:43.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1518356480 bytes 2026-02-19T20:18:43.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1567291392 bytes 2026-02-19T20:18:43.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1599578112 bytes 2026-02-19T20:18:43.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1653027840 bytes 2026-02-19T20:18:43.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1735019520 bytes 2026-02-19T20:18:43.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1761212416 bytes 2026-02-19T20:18:43.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1775371264 bytes 2026-02-19T20:18:43.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1829250048 bytes 2026-02-19T20:18:44.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1864868864 bytes 2026-02-19T20:18:44.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1946558464 bytes 2026-02-19T20:18:44.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2000185344 bytes 2026-02-19T20:18:44.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2026674176 bytes 2026-02-19T20:18:44.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2097685504 bytes 2026-02-19T20:18:44.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2111995904 bytes 2026-02-19T20:18:44.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2184400896 bytes 2026-02-19T20:18:44.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2237736960 bytes 2026-02-19T20:18:44.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2264757248 bytes 2026-02-19T20:18:44.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2296794112 bytes 2026-02-19T20:18:44.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2357551104 bytes 2026-02-19T20:18:44.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2433093632 bytes 2026-02-19T20:18:44.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2503022592 bytes 2026-02-19T20:18:44.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2565517312 bytes 2026-02-19T20:18:44.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2617652224 bytes 2026-02-19T20:18:45.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2711348224 bytes 2026-02-19T20:18:45.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2728550400 bytes 2026-02-19T20:18:45.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2767490048 bytes 2026-02-19T20:18:45.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2782968832 bytes 2026-02-19T20:18:45.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2799149056 bytes 2026-02-19T20:18:45.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2856619008 bytes 2026-02-19T20:18:45.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2945040384 bytes 2026-02-19T20:18:45.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2989476864 bytes 2026-02-19T20:18:45.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3001987072 bytes 2026-02-19T20:18:45.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3038289920 bytes 2026-02-19T20:18:45.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3085964288 bytes 2026-02-19T20:18:45.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3112379392 bytes 2026-02-19T20:18:45.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3133163520 bytes 2026-02-19T20:18:45.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3181630464 bytes 2026-02-19T20:18:45.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3240895488 bytes 2026-02-19T20:18:45.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3310607360 bytes 2026-02-19T20:18:45.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3384766464 bytes 2026-02-19T20:18:45.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3414786048 bytes 2026-02-19T20:18:46.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3436833792 bytes 2026-02-19T20:18:46.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3485615104 bytes 2026-02-19T20:18:46.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3503550464 bytes 2026-02-19T20:18:46.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3530607616 bytes 2026-02-19T20:18:46.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3540858880 bytes 2026-02-19T20:18:46.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3565175808 bytes 2026-02-19T20:18:46.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3607596032 bytes 2026-02-19T20:18:46.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3650186240 bytes 2026-02-19T20:18:46.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3668097024 bytes 2026-02-19T20:18:46.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3729353728 bytes 2026-02-19T20:18:46.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3778594816 bytes 2026-02-19T20:18:46.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3789206528 bytes 2026-02-19T20:18:46.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3869715456 bytes 2026-02-19T20:18:46.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3892810752 bytes 2026-02-19T20:18:46.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3924388864 bytes 2026-02-19T20:18:46.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3974048768 bytes 2026-02-19T20:18:46.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4065794048 bytes 2026-02-19T20:18:46.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4108792832 bytes 2026-02-19T20:18:47.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4141875200 bytes 2026-02-19T20:18:47.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4177432576 bytes 2026-02-19T20:18:47.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4226643968 bytes 2026-02-19T20:18:47.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4305500160 bytes 2026-02-19T20:18:47.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4351394816 bytes 2026-02-19T20:18:47.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4396921856 bytes 2026-02-19T20:18:47.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4419096576 bytes 2026-02-19T20:18:47.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4419690496 bytes 2026-02-19T20:18:47.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4509423616 bytes 2026-02-19T20:18:47.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4534329344 bytes 2026-02-19T20:18:47.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4535560192 bytes 2026-02-19T20:18:47.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4576464896 bytes 2026-02-19T20:18:47.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4582278144 bytes 2026-02-19T20:18:47.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4588490752 bytes 2026-02-19T20:18:47.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4630539264 bytes 2026-02-19T20:18:47.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4658489344 bytes 2026-02-19T20:18:47.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4693307392 bytes 2026-02-19T20:18:47.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4733989888 bytes 2026-02-19T20:18:47.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4745065472 bytes 2026-02-19T20:18:47.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4770565120 bytes 2026-02-19T20:18:47.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4792170496 bytes 2026-02-19T20:18:48.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4839969792 bytes 2026-02-19T20:18:48.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4877735936 bytes 2026-02-19T20:18:48.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4964227072 bytes 2026-02-19T20:18:48.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5019497472 bytes 2026-02-19T20:18:48.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5063636992 bytes 2026-02-19T20:18:48.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5066844160 bytes 2026-02-19T20:18:48.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5102597120 bytes 2026-02-19T20:18:48.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5201410048 bytes 2026-02-19T20:18:48.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5245308928 bytes 2026-02-19T20:18:48.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5263130624 bytes 2026-02-19T20:18:48.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5295803392 bytes 2026-02-19T20:18:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5348346880 bytes 2026-02-19T20:18:48.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5389103104 bytes 2026-02-19T20:18:48.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5410739200 bytes 2026-02-19T20:18:48.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5525643264 bytes 2026-02-19T20:18:48.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5536545792 bytes 2026-02-19T20:18:48.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5539963904 bytes 2026-02-19T20:18:49.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5650058240 bytes 2026-02-19T20:18:49.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5678039040 bytes 2026-02-19T20:18:49.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5727547392 bytes 2026-02-19T20:18:49.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5728163840 bytes 2026-02-19T20:18:49.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5757279232 bytes 2026-02-19T20:18:49.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5757901824 bytes 2026-02-19T20:18:49.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5812387840 bytes 2026-02-19T20:18:49.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5829307392 bytes 2026-02-19T20:18:49.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5877727232 bytes 2026-02-19T20:18:49.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5927196672 bytes 2026-02-19T20:18:49.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5982400512 bytes 2026-02-19T20:18:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6039254016 bytes 2026-02-19T20:18:49.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6073929728 bytes 2026-02-19T20:18:49.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6096235520 bytes 2026-02-19T20:18:49.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6186375168 bytes 2026-02-19T20:18:49.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6233620480 bytes 2026-02-19T20:18:49.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6249918464 bytes 2026-02-19T20:18:49.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6284503040 bytes 2026-02-19T20:18:50.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6342900736 bytes 2026-02-19T20:18:50.049 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6343508992 bytes 2026-02-19T20:18:50.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6425300992 bytes 2026-02-19T20:18:50.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6438512640 bytes 2026-02-19T20:18:50.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6447691776 bytes 2026-02-19T20:18:50.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6497074176 bytes 2026-02-19T20:18:50.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6571725824 bytes 2026-02-19T20:18:50.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6605036544 bytes 2026-02-19T20:18:50.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6667918336 bytes 2026-02-19T20:18:50.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6715898880 bytes 2026-02-19T20:18:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6807503872 bytes 2026-02-19T20:18:50.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6872201216 bytes 2026-02-19T20:18:50.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6956601344 bytes 2026-02-19T20:18:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7007117312 bytes 2026-02-19T20:18:50.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7088583680 bytes 2026-02-19T20:18:50.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7117277184 bytes 2026-02-19T20:18:51.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7171246080 bytes 2026-02-19T20:18:51.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7199326208 bytes 2026-02-19T20:18:51.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7257744384 bytes 2026-02-19T20:18:51.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7268324352 bytes 2026-02-19T20:18:51.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7288197120 bytes 2026-02-19T20:18:51.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7350412288 bytes 2026-02-19T20:18:51.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7363546112 bytes 2026-02-19T20:18:51.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7419026432 bytes 2026-02-19T20:18:51.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7467422720 bytes 2026-02-19T20:18:51.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7481548800 bytes 2026-02-19T20:18:51.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7554678784 bytes 2026-02-19T20:18:51.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7620427776 bytes 2026-02-19T20:18:51.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7665604608 bytes 2026-02-19T20:18:51.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7748531200 bytes 2026-02-19T20:18:51.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7805399040 bytes 2026-02-19T20:18:51.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7885284352 bytes 2026-02-19T20:18:52.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7953512448 bytes 2026-02-19T20:18:52.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 8075600896 bytes 2026-02-19T20:18:52.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 8171823104 bytes 2026-02-19T20:18:52.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 8187607040 bytes 2026-02-19T20:18:52.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 8193455104 bytes 2026-02-19T20:18:52.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8254827520 bytes 2026-02-19T20:18:52.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8330783744 bytes 2026-02-19T20:18:52.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8402174976 bytes 2026-02-19T20:18:52.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8432689152 bytes 2026-02-19T20:18:52.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8454165504 bytes 2026-02-19T20:18:52.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8493834240 bytes 2026-02-19T20:18:52.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8525872128 bytes 2026-02-19T20:18:52.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8545182720 bytes 2026-02-19T20:18:52.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8561176576 bytes 2026-02-19T20:18:52.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8606588928 bytes 2026-02-19T20:18:52.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8650974208 bytes 2026-02-19T20:18:53.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8672713728 bytes 2026-02-19T20:18:53.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8699546624 bytes 2026-02-19T20:18:53.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8762947584 bytes 2026-02-19T20:18:53.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8831214592 bytes 2026-02-19T20:18:53.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8865438720 bytes 2026-02-19T20:18:53.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8890785792 bytes 2026-02-19T20:18:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8948502528 bytes 2026-02-19T20:18:53.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8963217408 bytes 2026-02-19T20:18:53.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8978636800 bytes 2026-02-19T20:18:53.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 9074746368 bytes 2026-02-19T20:18:53.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 9095101440 bytes 2026-02-19T20:18:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 9141465088 bytes 2026-02-19T20:18:53.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 9199314944 bytes 2026-02-19T20:18:53.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 9265629184 bytes 2026-02-19T20:18:53.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 9284402176 bytes 2026-02-19T20:18:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 9304617984 bytes 2026-02-19T20:18:53.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 9360276480 bytes 2026-02-19T20:18:53.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 9433265152 bytes 2026-02-19T20:18:54.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 9456861184 bytes 2026-02-19T20:18:54.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 9489935360 bytes 2026-02-19T20:18:54.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 9521347584 bytes 2026-02-19T20:18:54.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 9571201024 bytes 2026-02-19T20:18:54.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 9580005376 bytes 2026-02-19T20:18:54.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 9609057280 bytes 2026-02-19T20:18:54.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 9697711104 bytes 2026-02-19T20:18:54.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 9735257088 bytes 2026-02-19T20:18:54.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 9745141760 bytes 2026-02-19T20:18:54.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 9795091456 bytes 2026-02-19T20:18:54.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 9854830592 bytes 2026-02-19T20:18:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:18:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:18:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:18:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_5 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_5 snapshot 128M 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:18:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:54.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_5 128M 2026-02-19T20:18:54.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:54.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:54.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:54.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:54.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:18:54.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:54.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:18:54.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:18:54.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:18:54.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:18:54.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_5 2026-02-19T20:18:54.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:18:54.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:54.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:18:54.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:18:54.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_5 snapshot 2026-02-19T20:18:54.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:54.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:54.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:18:54.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:18:54.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_5 snapshot 2026-02-19T20:18:55.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:18:55.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_5 2026-02-19T20:18:55.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_5': 2026-02-19T20:18:55.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:18:55.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:18:55.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:18:55.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 129960d6e56a 2026-02-19T20:18:55.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.129960d6e56a 2026-02-19T20:18:55.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:18:55.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:18:55.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:18:55.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:18:55.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:54 2026 2026-02-19T20:18:55.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:54 2026 2026-02-19T20:18:55.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:54 2026 2026-02-19T20:18:55.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:18:55.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:18:55.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: e1d197e7-fd23-4005-8baf-4a772b01d8ee 2026-02-19T20:18:55.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:18:55.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:18:55.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_5 2026-02-19T20:18:55.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:18:55.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:18:55.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:18:55.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:18:55.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:18:55.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:18:55.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_5 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:18:55.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 55357440 bytes 2026-02-19T20:18:55.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 108403712 bytes 2026-02-19T20:18:55.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 166054912 bytes 2026-02-19T20:18:55.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 201870336 bytes 2026-02-19T20:18:55.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 215825408 bytes 2026-02-19T20:18:55.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 278897664 bytes 2026-02-19T20:18:55.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 299561984 bytes 2026-02-19T20:18:56.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 346187776 bytes 2026-02-19T20:18:56.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 355217408 bytes 2026-02-19T20:18:56.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 393455616 bytes 2026-02-19T20:18:56.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 394089472 bytes 2026-02-19T20:18:56.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 401363968 bytes 2026-02-19T20:18:56.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 431400960 bytes 2026-02-19T20:18:56.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 492264448 bytes 2026-02-19T20:18:56.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 546915328 bytes 2026-02-19T20:18:56.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 617849856 bytes 2026-02-19T20:18:56.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 683538432 bytes 2026-02-19T20:18:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 708618240 bytes 2026-02-19T20:18:56.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 766328832 bytes 2026-02-19T20:18:56.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 766916608 bytes 2026-02-19T20:18:56.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 813730816 bytes 2026-02-19T20:18:56.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 854387712 bytes 2026-02-19T20:18:56.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 891710464 bytes 2026-02-19T20:18:57.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 926718976 bytes 2026-02-19T20:18:57.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 928565248 bytes 2026-02-19T20:18:57.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 982407168 bytes 2026-02-19T20:18:57.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 983015424 bytes 2026-02-19T20:18:57.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1044825088 bytes 2026-02-19T20:18:57.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1094706176 bytes 2026-02-19T20:18:57.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1110566912 bytes 2026-02-19T20:18:57.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1151717376 bytes 2026-02-19T20:18:57.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1181961216 bytes 2026-02-19T20:18:57.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1190422528 bytes 2026-02-19T20:18:57.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1233908736 bytes 2026-02-19T20:18:57.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1331725312 bytes 2026-02-19T20:18:57.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1389110272 bytes 2026-02-19T20:18:57.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1445271552 bytes 2026-02-19T20:18:57.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1447058432 bytes 2026-02-19T20:18:57.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1477078016 bytes 2026-02-19T20:18:57.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1507335168 bytes 2026-02-19T20:18:57.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1550570496 bytes 2026-02-19T20:18:57.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1566776320 bytes 2026-02-19T20:18:58.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1604616192 bytes 2026-02-19T20:18:58.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1605228544 bytes 2026-02-19T20:18:58.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1644692480 bytes 2026-02-19T20:18:58.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1669850112 bytes 2026-02-19T20:18:58.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1705585664 bytes 2026-02-19T20:18:58.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1788034048 bytes 2026-02-19T20:18:58.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1788653568 bytes 2026-02-19T20:18:58.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1816260608 bytes 2026-02-19T20:18:58.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1841117184 bytes 2026-02-19T20:18:58.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 1921883136 bytes 2026-02-19T20:18:58.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 1941043200 bytes 2026-02-19T20:18:58.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 1972566016 bytes 2026-02-19T20:18:58.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2012200960 bytes 2026-02-19T20:18:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2064517120 bytes 2026-02-19T20:18:58.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2123189248 bytes 2026-02-19T20:18:58.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2169414656 bytes 2026-02-19T20:18:58.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2198935552 bytes 2026-02-19T20:18:58.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2235537408 bytes 2026-02-19T20:18:58.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2236157952 bytes 2026-02-19T20:18:58.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2251907072 bytes 2026-02-19T20:18:59.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2307630080 bytes 2026-02-19T20:18:59.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2308244480 bytes 2026-02-19T20:18:59.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2345454592 bytes 2026-02-19T20:18:59.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2393901056 bytes 2026-02-19T20:18:59.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2428886016 bytes 2026-02-19T20:18:59.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2496009216 bytes 2026-02-19T20:18:59.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2548368384 bytes 2026-02-19T20:18:59.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2591539200 bytes 2026-02-19T20:18:59.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2634694656 bytes 2026-02-19T20:18:59.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2685070336 bytes 2026-02-19T20:18:59.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2784283648 bytes 2026-02-19T20:18:59.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2840312832 bytes 2026-02-19T20:18:59.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2899445760 bytes 2026-02-19T20:18:59.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2900028416 bytes 2026-02-19T20:18:59.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 2947373056 bytes 2026-02-19T20:18:59.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 2981767168 bytes 2026-02-19T20:19:00.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3023782912 bytes 2026-02-19T20:19:00.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3024396288 bytes 2026-02-19T20:19:00.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3091219456 bytes 2026-02-19T20:19:00.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3125643264 bytes 2026-02-19T20:19:00.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3184476160 bytes 2026-02-19T20:19:00.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3287239680 bytes 2026-02-19T20:19:00.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3287865344 bytes 2026-02-19T20:19:00.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3303083008 bytes 2026-02-19T20:19:00.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3352193024 bytes 2026-02-19T20:19:00.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3436069888 bytes 2026-02-19T20:19:00.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3447291904 bytes 2026-02-19T20:19:00.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3483624448 bytes 2026-02-19T20:19:00.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3545502720 bytes 2026-02-19T20:19:00.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3546114048 bytes 2026-02-19T20:19:00.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3588353024 bytes 2026-02-19T20:19:00.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3647665152 bytes 2026-02-19T20:19:00.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3685352448 bytes 2026-02-19T20:19:00.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3703955456 bytes 2026-02-19T20:19:00.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3754771456 bytes 2026-02-19T20:19:01.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3774651392 bytes 2026-02-19T20:19:01.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3824406528 bytes 2026-02-19T20:19:01.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 3861450752 bytes 2026-02-19T20:19:01.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 3907070976 bytes 2026-02-19T20:19:01.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 3956959232 bytes 2026-02-19T20:19:01.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4002640896 bytes 2026-02-19T20:19:01.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4050862080 bytes 2026-02-19T20:19:01.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4071180288 bytes 2026-02-19T20:19:01.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4098381824 bytes 2026-02-19T20:19:01.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4122317824 bytes 2026-02-19T20:19:01.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4172433408 bytes 2026-02-19T20:19:01.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4257654784 bytes 2026-02-19T20:19:01.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4308721664 bytes 2026-02-19T20:19:01.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4355638272 bytes 2026-02-19T20:19:01.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4413020160 bytes 2026-02-19T20:19:01.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4442937344 bytes 2026-02-19T20:19:01.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4505922560 bytes 2026-02-19T20:19:02.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4528024576 bytes 2026-02-19T20:19:02.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4566632448 bytes 2026-02-19T20:19:02.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4644261888 bytes 2026-02-19T20:19:02.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4666055680 bytes 2026-02-19T20:19:02.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4714297344 bytes 2026-02-19T20:19:02.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4727700480 bytes 2026-02-19T20:19:02.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4798297088 bytes 2026-02-19T20:19:02.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4842009600 bytes 2026-02-19T20:19:02.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4925945856 bytes 2026-02-19T20:19:02.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4960915456 bytes 2026-02-19T20:19:02.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5016637440 bytes 2026-02-19T20:19:02.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5080548352 bytes 2026-02-19T20:19:02.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5123613696 bytes 2026-02-19T20:19:02.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5160814592 bytes 2026-02-19T20:19:02.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5193276416 bytes 2026-02-19T20:19:02.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5199828992 bytes 2026-02-19T20:19:02.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5250009088 bytes 2026-02-19T20:19:03.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5307921408 bytes 2026-02-19T20:19:03.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5366612992 bytes 2026-02-19T20:19:03.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5403816960 bytes 2026-02-19T20:19:03.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5413688320 bytes 2026-02-19T20:19:03.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5456683008 bytes 2026-02-19T20:19:03.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5533238272 bytes 2026-02-19T20:19:03.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5547998208 bytes 2026-02-19T20:19:03.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5577613312 bytes 2026-02-19T20:19:03.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5618278400 bytes 2026-02-19T20:19:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5668652032 bytes 2026-02-19T20:19:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5694979072 bytes 2026-02-19T20:19:03.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5776132096 bytes 2026-02-19T20:19:03.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5821162496 bytes 2026-02-19T20:19:03.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5859103744 bytes 2026-02-19T20:19:03.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5932634112 bytes 2026-02-19T20:19:03.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5952989184 bytes 2026-02-19T20:19:03.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5985483776 bytes 2026-02-19T20:19:04.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6019472384 bytes 2026-02-19T20:19:04.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6040762368 bytes 2026-02-19T20:19:04.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6103579648 bytes 2026-02-19T20:19:04.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6136384512 bytes 2026-02-19T20:19:04.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6154185728 bytes 2026-02-19T20:19:04.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6185454592 bytes 2026-02-19T20:19:04.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6213614592 bytes 2026-02-19T20:19:04.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6246556672 bytes 2026-02-19T20:19:04.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6279552000 bytes 2026-02-19T20:19:04.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6304462848 bytes 2026-02-19T20:19:04.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6343096320 bytes 2026-02-19T20:19:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6357991424 bytes 2026-02-19T20:19:04.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6382438400 bytes 2026-02-19T20:19:04.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6390749184 bytes 2026-02-19T20:19:04.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6391370752 bytes 2026-02-19T20:19:04.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6393922560 bytes 2026-02-19T20:19:04.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6487913472 bytes 2026-02-19T20:19:04.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6540692480 bytes 2026-02-19T20:19:04.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6567075840 bytes 2026-02-19T20:19:05.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6629752832 bytes 2026-02-19T20:19:05.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6670181376 bytes 2026-02-19T20:19:05.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6670809088 bytes 2026-02-19T20:19:05.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6752110592 bytes 2026-02-19T20:19:05.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 6780076032 bytes 2026-02-19T20:19:05.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 6797294592 bytes 2026-02-19T20:19:05.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 6871085056 bytes 2026-02-19T20:19:05.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 6918989824 bytes 2026-02-19T20:19:05.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 6971406336 bytes 2026-02-19T20:19:05.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7051796480 bytes 2026-02-19T20:19:05.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7091772416 bytes 2026-02-19T20:19:05.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7123894272 bytes 2026-02-19T20:19:05.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7124505600 bytes 2026-02-19T20:19:05.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7143315456 bytes 2026-02-19T20:19:05.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7193995264 bytes 2026-02-19T20:19:05.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7198045184 bytes 2026-02-19T20:19:05.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7245166592 bytes 2026-02-19T20:19:05.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7278777344 bytes 2026-02-19T20:19:06.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7396173824 bytes 2026-02-19T20:19:06.103 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7446177792 bytes 2026-02-19T20:19:06.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7466569728 bytes 2026-02-19T20:19:06.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7482888192 bytes 2026-02-19T20:19:06.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7492875264 bytes 2026-02-19T20:19:06.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7518097408 bytes 2026-02-19T20:19:06.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7583706112 bytes 2026-02-19T20:19:06.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7584297984 bytes 2026-02-19T20:19:06.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 7624432640 bytes 2026-02-19T20:19:06.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 7672647680 bytes 2026-02-19T20:19:06.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 7758263296 bytes 2026-02-19T20:19:06.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 7803727872 bytes 2026-02-19T20:19:06.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 7850329088 bytes 2026-02-19T20:19:06.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 7940386816 bytes 2026-02-19T20:19:06.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8024130560 bytes 2026-02-19T20:19:06.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8047076352 bytes 2026-02-19T20:19:06.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8128643072 bytes 2026-02-19T20:19:07.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8180919296 bytes 2026-02-19T20:19:07.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8202436608 bytes 2026-02-19T20:19:07.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8218356736 bytes 2026-02-19T20:19:07.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8233213952 bytes 2026-02-19T20:19:07.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8242359296 bytes 2026-02-19T20:19:07.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8296206336 bytes 2026-02-19T20:19:07.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 8362999808 bytes 2026-02-19T20:19:07.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 8408790016 bytes 2026-02-19T20:19:07.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 8442708992 bytes 2026-02-19T20:19:07.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 8500645888 bytes 2026-02-19T20:19:07.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:19:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:19:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:19:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:19:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_6 2026-02-19T20:19:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_6 snapshot 128M 2026-02-19T20:19:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_6 128M 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:07.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:19:07.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:07.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:19:07.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:19:07.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:19:07.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:07.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_6 2026-02-19T20:19:07.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:19:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:19:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_6 snapshot 2026-02-19T20:19:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:19:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:07.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_6 snapshot 2026-02-19T20:19:08.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:19:08.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_6 2026-02-19T20:19:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_6': 2026-02-19T20:19:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:19:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:19:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:19:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 12a51032d209 2026-02-19T20:19:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.12a51032d209 2026-02-19T20:19:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:19:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:19:08.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:19:08.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:19:08.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:07 2026 2026-02-19T20:19:08.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:07 2026 2026-02-19T20:19:08.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:07 2026 2026-02-19T20:19:08.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:19:08.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:19:08.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 3465736d-6dc5-4652-9dad-a7ea19b6ea48 2026-02-19T20:19:08.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:19:08.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:19:08.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_6 2026-02-19T20:19:08.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:08.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:08.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:19:08.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:19:08.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=5 2026-02-19T20:19:08.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:19:08.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_6 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:19:08.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 28103680 bytes 2026-02-19T20:19:08.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 99203072 bytes 2026-02-19T20:19:08.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 116280320 bytes 2026-02-19T20:19:08.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 153417728 bytes 2026-02-19T20:19:08.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 197728256 bytes 2026-02-19T20:19:08.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 251066368 bytes 2026-02-19T20:19:08.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 252584960 bytes 2026-02-19T20:19:09.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 274250752 bytes 2026-02-19T20:19:09.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 332451840 bytes 2026-02-19T20:19:09.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 370402304 bytes 2026-02-19T20:19:09.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 376650752 bytes 2026-02-19T20:19:09.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 417610752 bytes 2026-02-19T20:19:09.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 419994624 bytes 2026-02-19T20:19:09.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 453184512 bytes 2026-02-19T20:19:09.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 460961792 bytes 2026-02-19T20:19:09.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 509112320 bytes 2026-02-19T20:19:09.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 545152000 bytes 2026-02-19T20:19:09.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 596836352 bytes 2026-02-19T20:19:09.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 654567424 bytes 2026-02-19T20:19:09.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 689743872 bytes 2026-02-19T20:19:09.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 715626496 bytes 2026-02-19T20:19:09.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 736256000 bytes 2026-02-19T20:19:09.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 780076032 bytes 2026-02-19T20:19:09.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 836554752 bytes 2026-02-19T20:19:09.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 932662272 bytes 2026-02-19T20:19:10.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 958522368 bytes 2026-02-19T20:19:10.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1069888512 bytes 2026-02-19T20:19:10.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1130924032 bytes 2026-02-19T20:19:10.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1194823680 bytes 2026-02-19T20:19:10.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1232687104 bytes 2026-02-19T20:19:10.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1277954048 bytes 2026-02-19T20:19:10.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1305586688 bytes 2026-02-19T20:19:10.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1316025344 bytes 2026-02-19T20:19:10.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1339342848 bytes 2026-02-19T20:19:10.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1368572928 bytes 2026-02-19T20:19:10.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1390269440 bytes 2026-02-19T20:19:10.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1453064192 bytes 2026-02-19T20:19:10.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1478690816 bytes 2026-02-19T20:19:10.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1528966144 bytes 2026-02-19T20:19:10.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1558885376 bytes 2026-02-19T20:19:11.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1606273024 bytes 2026-02-19T20:19:11.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1653544960 bytes 2026-02-19T20:19:11.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1756635136 bytes 2026-02-19T20:19:11.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1765859328 bytes 2026-02-19T20:19:11.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1798682624 bytes 2026-02-19T20:19:11.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1829285888 bytes 2026-02-19T20:19:11.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1875208192 bytes 2026-02-19T20:19:11.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1917837312 bytes 2026-02-19T20:19:11.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1932755968 bytes 2026-02-19T20:19:11.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2027192320 bytes 2026-02-19T20:19:11.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2066110464 bytes 2026-02-19T20:19:11.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2089425920 bytes 2026-02-19T20:19:11.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2109262848 bytes 2026-02-19T20:19:11.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2146824192 bytes 2026-02-19T20:19:11.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2183720960 bytes 2026-02-19T20:19:11.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2200087552 bytes 2026-02-19T20:19:11.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2212068352 bytes 2026-02-19T20:19:11.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2241534976 bytes 2026-02-19T20:19:12.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2297489408 bytes 2026-02-19T20:19:12.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2358060032 bytes 2026-02-19T20:19:12.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2428208128 bytes 2026-02-19T20:19:12.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2452022272 bytes 2026-02-19T20:19:12.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2456328192 bytes 2026-02-19T20:19:12.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2456933376 bytes 2026-02-19T20:19:12.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2494160896 bytes 2026-02-19T20:19:12.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2511775744 bytes 2026-02-19T20:19:12.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2562204672 bytes 2026-02-19T20:19:12.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2617401344 bytes 2026-02-19T20:19:12.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2625208320 bytes 2026-02-19T20:19:12.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2680790016 bytes 2026-02-19T20:19:12.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2714129408 bytes 2026-02-19T20:19:12.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2758828032 bytes 2026-02-19T20:19:12.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2759434240 bytes 2026-02-19T20:19:12.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2833700864 bytes 2026-02-19T20:19:12.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2857772032 bytes 2026-02-19T20:19:12.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2879510528 bytes 2026-02-19T20:19:12.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 2891344896 bytes 2026-02-19T20:19:13.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 2926876672 bytes 2026-02-19T20:19:13.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 2974294016 bytes 2026-02-19T20:19:13.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 2993522688 bytes 2026-02-19T20:19:13.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3024024576 bytes 2026-02-19T20:19:13.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3041214464 bytes 2026-02-19T20:19:13.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3101654016 bytes 2026-02-19T20:19:13.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3106345984 bytes 2026-02-19T20:19:13.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3111710720 bytes 2026-02-19T20:19:13.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3149697024 bytes 2026-02-19T20:19:13.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3160871936 bytes 2026-02-19T20:19:13.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3172067328 bytes 2026-02-19T20:19:13.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:19:13.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:19:13.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:19:13.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:13.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_7 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_7 snapshot 128M 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:13.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_7 128M 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_7 2026-02-19T20:19:13.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:19:13.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:13.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:19:13.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:13.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_7 snapshot 2026-02-19T20:19:13.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:13.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:13.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:19:13.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:13.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_7 snapshot 2026-02-19T20:19:14.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:19:14.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_7 2026-02-19T20:19:14.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_7': 2026-02-19T20:19:14.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:19:14.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:19:14.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:19:14.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 12b15b953145 2026-02-19T20:19:14.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.12b15b953145 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:13 2026 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:13 2026 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:13 2026 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 5770e33c-f75b-4af2-9f16-1bf9dbb1a48f 2026-02-19T20:19:14.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:19:14.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:19:14.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_7 2026-02-19T20:19:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:19:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:19:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=11 2026-02-19T20:19:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:19:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_7 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:19:14.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 38814720 bytes 2026-02-19T20:19:14.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 62769152 bytes 2026-02-19T20:19:14.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 94996480 bytes 2026-02-19T20:19:14.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 129771520 bytes 2026-02-19T20:19:14.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 193780736 bytes 2026-02-19T20:19:14.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 254556160 bytes 2026-02-19T20:19:15.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 309250048 bytes 2026-02-19T20:19:15.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 358482944 bytes 2026-02-19T20:19:15.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 398305280 bytes 2026-02-19T20:19:15.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 464973824 bytes 2026-02-19T20:19:15.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 465605632 bytes 2026-02-19T20:19:15.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 511981568 bytes 2026-02-19T20:19:15.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 525915136 bytes 2026-02-19T20:19:15.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 557145088 bytes 2026-02-19T20:19:15.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 627296256 bytes 2026-02-19T20:19:15.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 766459904 bytes 2026-02-19T20:19:15.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 808212480 bytes 2026-02-19T20:19:15.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 886701056 bytes 2026-02-19T20:19:15.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 924624896 bytes 2026-02-19T20:19:15.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 946774016 bytes 2026-02-19T20:19:15.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 993702912 bytes 2026-02-19T20:19:15.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1048876032 bytes 2026-02-19T20:19:16.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1131822080 bytes 2026-02-19T20:19:16.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1167879168 bytes 2026-02-19T20:19:16.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1217397760 bytes 2026-02-19T20:19:16.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1257827328 bytes 2026-02-19T20:19:16.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1292290048 bytes 2026-02-19T20:19:16.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1402150912 bytes 2026-02-19T20:19:16.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1460844544 bytes 2026-02-19T20:19:16.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1491977216 bytes 2026-02-19T20:19:16.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1523841024 bytes 2026-02-19T20:19:16.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1550715904 bytes 2026-02-19T20:19:16.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1591244800 bytes 2026-02-19T20:19:16.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1625487360 bytes 2026-02-19T20:19:16.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1666944000 bytes 2026-02-19T20:19:16.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1730333696 bytes 2026-02-19T20:19:16.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1758396416 bytes 2026-02-19T20:19:16.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1834023936 bytes 2026-02-19T20:19:17.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1896647680 bytes 2026-02-19T20:19:17.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 2009109504 bytes 2026-02-19T20:19:17.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 2133071872 bytes 2026-02-19T20:19:17.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 2194279424 bytes 2026-02-19T20:19:17.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2237757440 bytes 2026-02-19T20:19:17.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2264913920 bytes 2026-02-19T20:19:17.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2307663872 bytes 2026-02-19T20:19:17.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2329478144 bytes 2026-02-19T20:19:17.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2340940800 bytes 2026-02-19T20:19:17.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2455434240 bytes 2026-02-19T20:19:17.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2497298432 bytes 2026-02-19T20:19:17.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2543282176 bytes 2026-02-19T20:19:17.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2630728704 bytes 2026-02-19T20:19:17.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2707644416 bytes 2026-02-19T20:19:17.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2723911680 bytes 2026-02-19T20:19:18.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2742913024 bytes 2026-02-19T20:19:18.077 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2771230720 bytes 2026-02-19T20:19:18.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2850655232 bytes 2026-02-19T20:19:18.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2909645824 bytes 2026-02-19T20:19:18.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2970983424 bytes 2026-02-19T20:19:18.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 3078555648 bytes 2026-02-19T20:19:18.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 3165040640 bytes 2026-02-19T20:19:18.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 3165663232 bytes 2026-02-19T20:19:18.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 3193196544 bytes 2026-02-19T20:19:18.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 3210243072 bytes 2026-02-19T20:19:18.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 3267680256 bytes 2026-02-19T20:19:18.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3302958080 bytes 2026-02-19T20:19:18.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3381334016 bytes 2026-02-19T20:19:18.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3398337536 bytes 2026-02-19T20:19:18.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3418488832 bytes 2026-02-19T20:19:18.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3427842048 bytes 2026-02-19T20:19:18.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3461623808 bytes 2026-02-19T20:19:19.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3480264704 bytes 2026-02-19T20:19:19.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3532034048 bytes 2026-02-19T20:19:19.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3598731264 bytes 2026-02-19T20:19:19.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3619600384 bytes 2026-02-19T20:19:19.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3707718656 bytes 2026-02-19T20:19:19.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3731894272 bytes 2026-02-19T20:19:19.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3813789696 bytes 2026-02-19T20:19:19.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3843343360 bytes 2026-02-19T20:19:19.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3896495104 bytes 2026-02-19T20:19:19.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3974960128 bytes 2026-02-19T20:19:19.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 4021684224 bytes 2026-02-19T20:19:19.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 4042504192 bytes 2026-02-19T20:19:20.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 4132171776 bytes 2026-02-19T20:19:20.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 4160604160 bytes 2026-02-19T20:19:20.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 4226379776 bytes 2026-02-19T20:19:20.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 4290990080 bytes 2026-02-19T20:19:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4348701696 bytes 2026-02-19T20:19:20.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4456750080 bytes 2026-02-19T20:19:20.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4464783360 bytes 2026-02-19T20:19:20.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4521832448 bytes 2026-02-19T20:19:20.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4552232960 bytes 2026-02-19T20:19:20.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4575229952 bytes 2026-02-19T20:19:20.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4590044160 bytes 2026-02-19T20:19:20.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4617232384 bytes 2026-02-19T20:19:20.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4633656320 bytes 2026-02-19T20:19:20.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4668103680 bytes 2026-02-19T20:19:20.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4741010432 bytes 2026-02-19T20:19:21.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4792979456 bytes 2026-02-19T20:19:21.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4852721664 bytes 2026-02-19T20:19:21.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4905377792 bytes 2026-02-19T20:19:21.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4977304576 bytes 2026-02-19T20:19:21.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4983075840 bytes 2026-02-19T20:19:21.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 5049871360 bytes 2026-02-19T20:19:21.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 5073489920 bytes 2026-02-19T20:19:21.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 5141751808 bytes 2026-02-19T20:19:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 5160375296 bytes 2026-02-19T20:19:21.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 5240158208 bytes 2026-02-19T20:19:21.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 5251710976 bytes 2026-02-19T20:19:21.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 5278674944 bytes 2026-02-19T20:19:21.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5284407296 bytes 2026-02-19T20:19:21.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5315569664 bytes 2026-02-19T20:19:21.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5316195328 bytes 2026-02-19T20:19:21.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5379432448 bytes 2026-02-19T20:19:21.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5403597824 bytes 2026-02-19T20:19:22.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5455879168 bytes 2026-02-19T20:19:22.049 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5484686336 bytes 2026-02-19T20:19:22.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5537805312 bytes 2026-02-19T20:19:22.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5572659200 bytes 2026-02-19T20:19:22.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5628016640 bytes 2026-02-19T20:19:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5713883136 bytes 2026-02-19T20:19:22.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5746525184 bytes 2026-02-19T20:19:22.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5785154560 bytes 2026-02-19T20:19:22.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5836435456 bytes 2026-02-19T20:19:22.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5846093824 bytes 2026-02-19T20:19:22.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5886657536 bytes 2026-02-19T20:19:22.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5890064384 bytes 2026-02-19T20:19:22.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5918877696 bytes 2026-02-19T20:19:22.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5969058816 bytes 2026-02-19T20:19:22.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5994872832 bytes 2026-02-19T20:19:22.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5998764032 bytes 2026-02-19T20:19:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 6031856640 bytes 2026-02-19T20:19:22.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 6048013312 bytes 2026-02-19T20:19:22.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 6057858048 bytes 2026-02-19T20:19:23.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6109898752 bytes 2026-02-19T20:19:23.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6154688512 bytes 2026-02-19T20:19:23.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6161075200 bytes 2026-02-19T20:19:23.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6176216064 bytes 2026-02-19T20:19:23.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6220370944 bytes 2026-02-19T20:19:23.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6227295232 bytes 2026-02-19T20:19:23.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6281480192 bytes 2026-02-19T20:19:23.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6296722432 bytes 2026-02-19T20:19:23.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6320807936 bytes 2026-02-19T20:19:23.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6341196800 bytes 2026-02-19T20:19:23.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6447003648 bytes 2026-02-19T20:19:23.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6552280064 bytes 2026-02-19T20:19:23.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6699847680 bytes 2026-02-19T20:19:23.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6726897664 bytes 2026-02-19T20:19:23.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6771283968 bytes 2026-02-19T20:19:23.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6789239808 bytes 2026-02-19T20:19:23.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6804875264 bytes 2026-02-19T20:19:24.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6844100608 bytes 2026-02-19T20:19:24.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6886483968 bytes 2026-02-19T20:19:24.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6964353024 bytes 2026-02-19T20:19:24.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6969467904 bytes 2026-02-19T20:19:24.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 7016329216 bytes 2026-02-19T20:19:24.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7036633088 bytes 2026-02-19T20:19:24.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7051682816 bytes 2026-02-19T20:19:24.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7091138560 bytes 2026-02-19T20:19:24.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7113018368 bytes 2026-02-19T20:19:24.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7148589056 bytes 2026-02-19T20:19:24.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7171664896 bytes 2026-02-19T20:19:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7189652480 bytes 2026-02-19T20:19:24.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7218780160 bytes 2026-02-19T20:19:24.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7290061824 bytes 2026-02-19T20:19:24.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7322909696 bytes 2026-02-19T20:19:24.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7380600832 bytes 2026-02-19T20:19:24.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7430661120 bytes 2026-02-19T20:19:25.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7493623808 bytes 2026-02-19T20:19:25.103 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7558227968 bytes 2026-02-19T20:19:25.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7569226752 bytes 2026-02-19T20:19:25.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7589277696 bytes 2026-02-19T20:19:25.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7642895360 bytes 2026-02-19T20:19:25.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7679974400 bytes 2026-02-19T20:19:25.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7721239552 bytes 2026-02-19T20:19:25.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7782815744 bytes 2026-02-19T20:19:25.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7835456512 bytes 2026-02-19T20:19:25.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7875681280 bytes 2026-02-19T20:19:25.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:19:25.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:19:25.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_8 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_8 snapshot 128M 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_8 128M 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:19:25.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:19:25.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:25.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_8 2026-02-19T20:19:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:19:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:19:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_8 snapshot 2026-02-19T20:19:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:19:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_8 snapshot 2026-02-19T20:19:26.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:19:26.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_8 2026-02-19T20:19:26.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_8': 2026-02-19T20:19:26.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:19:26.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:19:26.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:19:26.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 12bde71aee54 2026-02-19T20:19:26.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.12bde71aee54 2026-02-19T20:19:26.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:19:26.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:19:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:19:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:19:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:25 2026 2026-02-19T20:19:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:25 2026 2026-02-19T20:19:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:25 2026 2026-02-19T20:19:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:19:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:19:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: c9480ce4-7cc9-491d-9e71-9527ddc66985 2026-02-19T20:19:26.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:19:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:19:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_8 2026-02-19T20:19:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:19:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:19:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=14 2026-02-19T20:19:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:19:26.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_8 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:19:26.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 24545280 bytes 2026-02-19T20:19:26.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 49357824 bytes 2026-02-19T20:19:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 66091008 bytes 2026-02-19T20:19:26.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 110202880 bytes 2026-02-19T20:19:26.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 201348096 bytes 2026-02-19T20:19:26.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 234761216 bytes 2026-02-19T20:19:26.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 260096000 bytes 2026-02-19T20:19:27.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 331860992 bytes 2026-02-19T20:19:27.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 393459712 bytes 2026-02-19T20:19:27.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 451105792 bytes 2026-02-19T20:19:27.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 495932416 bytes 2026-02-19T20:19:27.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 511579136 bytes 2026-02-19T20:19:27.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 547917824 bytes 2026-02-19T20:19:27.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 598575104 bytes 2026-02-19T20:19:27.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 677865472 bytes 2026-02-19T20:19:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 723499008 bytes 2026-02-19T20:19:27.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 773821440 bytes 2026-02-19T20:19:27.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 871695360 bytes 2026-02-19T20:19:27.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 904310784 bytes 2026-02-19T20:19:27.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 948791296 bytes 2026-02-19T20:19:27.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 999805952 bytes 2026-02-19T20:19:28.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1014298624 bytes 2026-02-19T20:19:28.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1076837376 bytes 2026-02-19T20:19:28.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1090469888 bytes 2026-02-19T20:19:28.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1091086336 bytes 2026-02-19T20:19:28.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1122723840 bytes 2026-02-19T20:19:28.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1158673408 bytes 2026-02-19T20:19:28.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1179886592 bytes 2026-02-19T20:19:28.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1246752768 bytes 2026-02-19T20:19:28.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1273439232 bytes 2026-02-19T20:19:28.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1305933824 bytes 2026-02-19T20:19:28.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1367498752 bytes 2026-02-19T20:19:28.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1406269440 bytes 2026-02-19T20:19:28.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1444597760 bytes 2026-02-19T20:19:28.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1511284736 bytes 2026-02-19T20:19:28.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1580980224 bytes 2026-02-19T20:19:28.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1612867584 bytes 2026-02-19T20:19:28.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1646558208 bytes 2026-02-19T20:19:28.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1671197696 bytes 2026-02-19T20:19:29.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1699278848 bytes 2026-02-19T20:19:29.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1722764288 bytes 2026-02-19T20:19:29.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1780369408 bytes 2026-02-19T20:19:29.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1822015488 bytes 2026-02-19T20:19:29.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1911435264 bytes 2026-02-19T20:19:29.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2028983296 bytes 2026-02-19T20:19:29.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2085263360 bytes 2026-02-19T20:19:29.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2124227584 bytes 2026-02-19T20:19:29.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2155212800 bytes 2026-02-19T20:19:29.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2194992128 bytes 2026-02-19T20:19:29.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2281765888 bytes 2026-02-19T20:19:29.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2312994816 bytes 2026-02-19T20:19:29.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2331880448 bytes 2026-02-19T20:19:29.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2380581888 bytes 2026-02-19T20:19:29.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2449552384 bytes 2026-02-19T20:19:30.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2516510720 bytes 2026-02-19T20:19:30.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2539808768 bytes 2026-02-19T20:19:30.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2553985024 bytes 2026-02-19T20:19:30.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2583191552 bytes 2026-02-19T20:19:30.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2647957504 bytes 2026-02-19T20:19:30.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2676315136 bytes 2026-02-19T20:19:30.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2698830848 bytes 2026-02-19T20:19:30.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2711110656 bytes 2026-02-19T20:19:30.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2802485248 bytes 2026-02-19T20:19:30.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2820690944 bytes 2026-02-19T20:19:30.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2908433408 bytes 2026-02-19T20:19:30.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2949779456 bytes 2026-02-19T20:19:30.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3018314752 bytes 2026-02-19T20:19:30.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3055482880 bytes 2026-02-19T20:19:30.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3096086528 bytes 2026-02-19T20:19:30.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3175880704 bytes 2026-02-19T20:19:31.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3226706944 bytes 2026-02-19T20:19:31.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3251762176 bytes 2026-02-19T20:19:31.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3309907968 bytes 2026-02-19T20:19:31.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3322429440 bytes 2026-02-19T20:19:31.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3387345920 bytes 2026-02-19T20:19:31.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3449977856 bytes 2026-02-19T20:19:31.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3466089472 bytes 2026-02-19T20:19:31.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3518215168 bytes 2026-02-19T20:19:31.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3586252800 bytes 2026-02-19T20:19:31.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3623298048 bytes 2026-02-19T20:19:31.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3640635392 bytes 2026-02-19T20:19:31.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3658761216 bytes 2026-02-19T20:19:31.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3705631744 bytes 2026-02-19T20:19:31.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3732445184 bytes 2026-02-19T20:19:31.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3769986048 bytes 2026-02-19T20:19:31.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3787270144 bytes 2026-02-19T20:19:31.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3830810624 bytes 2026-02-19T20:19:32.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3872620544 bytes 2026-02-19T20:19:32.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3899746304 bytes 2026-02-19T20:19:32.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3904193536 bytes 2026-02-19T20:19:32.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3964325888 bytes 2026-02-19T20:19:32.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4001013760 bytes 2026-02-19T20:19:32.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4048500736 bytes 2026-02-19T20:19:32.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4141786112 bytes 2026-02-19T20:19:32.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4151968768 bytes 2026-02-19T20:19:32.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4209266688 bytes 2026-02-19T20:19:32.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4247019520 bytes 2026-02-19T20:19:32.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4298700800 bytes 2026-02-19T20:19:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4316152832 bytes 2026-02-19T20:19:32.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4328086528 bytes 2026-02-19T20:19:32.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4377218048 bytes 2026-02-19T20:19:32.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4400958464 bytes 2026-02-19T20:19:32.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4436832256 bytes 2026-02-19T20:19:32.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4500473856 bytes 2026-02-19T20:19:33.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4540557312 bytes 2026-02-19T20:19:33.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4585740288 bytes 2026-02-19T20:19:33.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4606379008 bytes 2026-02-19T20:19:33.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4683617280 bytes 2026-02-19T20:19:33.292 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4737698816 bytes 2026-02-19T20:19:33.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4835092480 bytes 2026-02-19T20:19:33.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4934046720 bytes 2026-02-19T20:19:33.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4979345408 bytes 2026-02-19T20:19:33.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4995902464 bytes 2026-02-19T20:19:33.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4999169024 bytes 2026-02-19T20:19:33.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5038855168 bytes 2026-02-19T20:19:33.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5106460672 bytes 2026-02-19T20:19:33.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5152976896 bytes 2026-02-19T20:19:33.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5252325376 bytes 2026-02-19T20:19:33.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5299033088 bytes 2026-02-19T20:19:34.083 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5377358848 bytes 2026-02-19T20:19:34.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5431867392 bytes 2026-02-19T20:19:34.230 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5489972224 bytes 2026-02-19T20:19:34.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5526087680 bytes 2026-02-19T20:19:34.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5527615488 bytes 2026-02-19T20:19:34.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5552822272 bytes 2026-02-19T20:19:34.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5584850944 bytes 2026-02-19T20:19:34.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5622876160 bytes 2026-02-19T20:19:34.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5648053248 bytes 2026-02-19T20:19:34.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5731402752 bytes 2026-02-19T20:19:34.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5827307520 bytes 2026-02-19T20:19:34.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5879122944 bytes 2026-02-19T20:19:34.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5935572992 bytes 2026-02-19T20:19:34.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 6016006144 bytes 2026-02-19T20:19:35.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6058919936 bytes 2026-02-19T20:19:35.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6166103040 bytes 2026-02-19T20:19:35.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6219100160 bytes 2026-02-19T20:19:35.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6221414400 bytes 2026-02-19T20:19:35.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6259936256 bytes 2026-02-19T20:19:35.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6260554752 bytes 2026-02-19T20:19:35.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6261157888 bytes 2026-02-19T20:19:35.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6288272384 bytes 2026-02-19T20:19:35.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6313948160 bytes 2026-02-19T20:19:35.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6328650752 bytes 2026-02-19T20:19:35.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6371799040 bytes 2026-02-19T20:19:35.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6409375744 bytes 2026-02-19T20:19:35.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6441290752 bytes 2026-02-19T20:19:35.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6507939840 bytes 2026-02-19T20:19:35.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6552810496 bytes 2026-02-19T20:19:35.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6565291008 bytes 2026-02-19T20:19:35.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6604998656 bytes 2026-02-19T20:19:35.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6646831104 bytes 2026-02-19T20:19:36.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6689768448 bytes 2026-02-19T20:19:36.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6702530560 bytes 2026-02-19T20:19:36.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6736966656 bytes 2026-02-19T20:19:36.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6778513408 bytes 2026-02-19T20:19:36.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6799538176 bytes 2026-02-19T20:19:36.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6843487232 bytes 2026-02-19T20:19:36.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6861457408 bytes 2026-02-19T20:19:36.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6882187264 bytes 2026-02-19T20:19:36.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6926171136 bytes 2026-02-19T20:19:36.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6955814912 bytes 2026-02-19T20:19:36.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7022344192 bytes 2026-02-19T20:19:36.591 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:19:36.589+0000 7f5bf91cc640 -1 reset not still connected to 0x570bad0 2026-02-19T20:19:36.592 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:19:36.589+0000 7f5bf91cc640 -1 reset not still connected to 0x570bc70 2026-02-19T20:19:36.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7075683328 bytes 2026-02-19T20:19:36.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7110317056 bytes 2026-02-19T20:19:36.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7208332288 bytes 2026-02-19T20:19:36.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7274737664 bytes 2026-02-19T20:19:36.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7340099584 bytes 2026-02-19T20:19:37.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7372218368 bytes 2026-02-19T20:19:37.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7386815488 bytes 2026-02-19T20:19:37.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7413974016 bytes 2026-02-19T20:19:37.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7451811840 bytes 2026-02-19T20:19:37.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7513399296 bytes 2026-02-19T20:19:37.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7579890688 bytes 2026-02-19T20:19:37.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7585018880 bytes 2026-02-19T20:19:37.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7629700096 bytes 2026-02-19T20:19:37.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7658132480 bytes 2026-02-19T20:19:37.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7694405632 bytes 2026-02-19T20:19:37.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7726960640 bytes 2026-02-19T20:19:37.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7799957504 bytes 2026-02-19T20:19:37.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7822825472 bytes 2026-02-19T20:19:37.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7876346880 bytes 2026-02-19T20:19:37.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7927876608 bytes 2026-02-19T20:19:37.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7957668864 bytes 2026-02-19T20:19:37.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7974860800 bytes 2026-02-19T20:19:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 8017705984 bytes 2026-02-19T20:19:38.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8078880768 bytes 2026-02-19T20:19:38.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8172424192 bytes 2026-02-19T20:19:38.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8232967168 bytes 2026-02-19T20:19:38.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8240304128 bytes 2026-02-19T20:19:38.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8294479872 bytes 2026-02-19T20:19:38.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8328673280 bytes 2026-02-19T20:19:38.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8341209088 bytes 2026-02-19T20:19:38.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8370049024 bytes 2026-02-19T20:19:38.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8395575296 bytes 2026-02-19T20:19:38.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8437415936 bytes 2026-02-19T20:19:38.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8489480192 bytes 2026-02-19T20:19:38.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8536898560 bytes 2026-02-19T20:19:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8595503104 bytes 2026-02-19T20:19:38.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8615666688 bytes 2026-02-19T20:19:38.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8635744256 bytes 2026-02-19T20:19:39.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8704274432 bytes 2026-02-19T20:19:39.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8763216896 bytes 2026-02-19T20:19:39.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8834114560 bytes 2026-02-19T20:19:39.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8852928512 bytes 2026-02-19T20:19:39.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8905737216 bytes 2026-02-19T20:19:39.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8983966720 bytes 2026-02-19T20:19:39.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 9023200256 bytes 2026-02-19T20:19:39.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 9042059264 bytes 2026-02-19T20:19:39.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 9076828160 bytes 2026-02-19T20:19:39.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 9093696512 bytes 2026-02-19T20:19:39.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 9146285056 bytes 2026-02-19T20:19:39.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 9214256128 bytes 2026-02-19T20:19:39.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 9217219584 bytes 2026-02-19T20:19:39.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 9264392192 bytes 2026-02-19T20:19:39.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 9273121792 bytes 2026-02-19T20:19:39.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 9306208256 bytes 2026-02-19T20:19:39.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 9326824448 bytes 2026-02-19T20:19:40.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 9363453952 bytes 2026-02-19T20:19:40.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 9386804224 bytes 2026-02-19T20:19:40.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 9415707648 bytes 2026-02-19T20:19:40.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 9484697600 bytes 2026-02-19T20:19:40.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 9509407744 bytes 2026-02-19T20:19:40.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 9573656576 bytes 2026-02-19T20:19:40.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 9670153216 bytes 2026-02-19T20:19:40.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 9682428928 bytes 2026-02-19T20:19:40.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22600 IOs, 9772723200 bytes 2026-02-19T20:19:40.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:19:40.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:19:40.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:19:40.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:40.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_9 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_9 snapshot 128M 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:19:40.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_9 128M 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_9 2026-02-19T20:19:40.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:19:40.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:40.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:19:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_9 snapshot 2026-02-19T20:19:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:19:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:40.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_9 snapshot 2026-02-19T20:19:41.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:19:41.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_9 2026-02-19T20:19:41.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_9': 2026-02-19T20:19:41.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:19:41.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:19:41.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:19:41.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 12c9140f77ea 2026-02-19T20:19:41.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.12c9140f77ea 2026-02-19T20:19:41.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:19:41.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:19:41.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:19:41.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:19:41.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:40 2026 2026-02-19T20:19:41.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:40 2026 2026-02-19T20:19:41.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:40 2026 2026-02-19T20:19:41.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:19:41.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:19:41.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 96e815a6-741f-4db1-bf35-a75f0477bd8e 2026-02-19T20:19:41.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:19:41.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:19:41.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_9 2026-02-19T20:19:41.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:19:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:19:41.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:19:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:19:41.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_9 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:19:41.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 31614976 bytes 2026-02-19T20:19:41.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 79587328 bytes 2026-02-19T20:19:41.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 104284160 bytes 2026-02-19T20:19:41.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 104884224 bytes 2026-02-19T20:19:41.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 200895488 bytes 2026-02-19T20:19:41.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 238139392 bytes 2026-02-19T20:19:42.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 301649920 bytes 2026-02-19T20:19:42.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 350770176 bytes 2026-02-19T20:19:42.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 383880192 bytes 2026-02-19T20:19:42.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 388401152 bytes 2026-02-19T20:19:42.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 442427392 bytes 2026-02-19T20:19:42.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 477817856 bytes 2026-02-19T20:19:42.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 517821440 bytes 2026-02-19T20:19:42.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 550309888 bytes 2026-02-19T20:19:42.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 574012416 bytes 2026-02-19T20:19:42.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 602363904 bytes 2026-02-19T20:19:42.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 656043008 bytes 2026-02-19T20:19:42.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 706114560 bytes 2026-02-19T20:19:42.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 722477056 bytes 2026-02-19T20:19:42.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 743671808 bytes 2026-02-19T20:19:42.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 751244288 bytes 2026-02-19T20:19:42.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 828603392 bytes 2026-02-19T20:19:42.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 872695808 bytes 2026-02-19T20:19:42.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 873299968 bytes 2026-02-19T20:19:43.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 928942080 bytes 2026-02-19T20:19:43.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 959508480 bytes 2026-02-19T20:19:43.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 981692416 bytes 2026-02-19T20:19:43.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1024495616 bytes 2026-02-19T20:19:43.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1037611008 bytes 2026-02-19T20:19:43.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1053214720 bytes 2026-02-19T20:19:43.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1100791808 bytes 2026-02-19T20:19:43.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1124133888 bytes 2026-02-19T20:19:43.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1124734976 bytes 2026-02-19T20:19:43.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1154510848 bytes 2026-02-19T20:19:43.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1197560832 bytes 2026-02-19T20:19:43.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1253505024 bytes 2026-02-19T20:19:43.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1271938048 bytes 2026-02-19T20:19:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1272544256 bytes 2026-02-19T20:19:43.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1297561600 bytes 2026-02-19T20:19:43.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1321762816 bytes 2026-02-19T20:19:43.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1333745664 bytes 2026-02-19T20:19:43.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1391409152 bytes 2026-02-19T20:19:43.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1404924928 bytes 2026-02-19T20:19:43.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1416130560 bytes 2026-02-19T20:19:44.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1458430976 bytes 2026-02-19T20:19:44.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1547777024 bytes 2026-02-19T20:19:44.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1635895296 bytes 2026-02-19T20:19:44.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1714019328 bytes 2026-02-19T20:19:44.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1744219136 bytes 2026-02-19T20:19:44.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1802094592 bytes 2026-02-19T20:19:44.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1813687296 bytes 2026-02-19T20:19:44.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 1864497152 bytes 2026-02-19T20:19:44.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 1890069504 bytes 2026-02-19T20:19:44.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 1912572928 bytes 2026-02-19T20:19:44.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 1944993792 bytes 2026-02-19T20:19:44.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2023853056 bytes 2026-02-19T20:19:44.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2083077120 bytes 2026-02-19T20:19:44.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2113021952 bytes 2026-02-19T20:19:44.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2180274176 bytes 2026-02-19T20:19:45.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2225495040 bytes 2026-02-19T20:19:45.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2265180160 bytes 2026-02-19T20:19:45.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2317676544 bytes 2026-02-19T20:19:45.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2423570432 bytes 2026-02-19T20:19:45.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2424177664 bytes 2026-02-19T20:19:45.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2476513280 bytes 2026-02-19T20:19:45.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2522616832 bytes 2026-02-19T20:19:45.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2530778112 bytes 2026-02-19T20:19:45.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2570131456 bytes 2026-02-19T20:19:45.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2648351744 bytes 2026-02-19T20:19:45.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2689166336 bytes 2026-02-19T20:19:45.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2727075840 bytes 2026-02-19T20:19:45.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2771414016 bytes 2026-02-19T20:19:45.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2795577344 bytes 2026-02-19T20:19:45.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2812220416 bytes 2026-02-19T20:19:45.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2848974848 bytes 2026-02-19T20:19:46.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2898176000 bytes 2026-02-19T20:19:46.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 2969797632 bytes 2026-02-19T20:19:46.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 2999364608 bytes 2026-02-19T20:19:46.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3111183360 bytes 2026-02-19T20:19:46.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3202229248 bytes 2026-02-19T20:19:46.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3232646144 bytes 2026-02-19T20:19:46.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3281308672 bytes 2026-02-19T20:19:46.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3388426240 bytes 2026-02-19T20:19:46.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3389038592 bytes 2026-02-19T20:19:46.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3442264064 bytes 2026-02-19T20:19:46.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3481078784 bytes 2026-02-19T20:19:46.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3544998912 bytes 2026-02-19T20:19:46.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3603190784 bytes 2026-02-19T20:19:46.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3625758720 bytes 2026-02-19T20:19:47.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3677443072 bytes 2026-02-19T20:19:47.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3732439040 bytes 2026-02-19T20:19:47.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3787996160 bytes 2026-02-19T20:19:47.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3823674368 bytes 2026-02-19T20:19:47.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3869064192 bytes 2026-02-19T20:19:47.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3873474560 bytes 2026-02-19T20:19:47.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3952104448 bytes 2026-02-19T20:19:47.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3968067584 bytes 2026-02-19T20:19:47.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4088779776 bytes 2026-02-19T20:19:47.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4147389440 bytes 2026-02-19T20:19:47.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4171601920 bytes 2026-02-19T20:19:48.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4204999680 bytes 2026-02-19T20:19:48.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4258413568 bytes 2026-02-19T20:19:48.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4354095104 bytes 2026-02-19T20:19:48.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4419426304 bytes 2026-02-19T20:19:48.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4460154880 bytes 2026-02-19T20:19:48.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4527232000 bytes 2026-02-19T20:19:48.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4618974208 bytes 2026-02-19T20:19:48.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4663211008 bytes 2026-02-19T20:19:48.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4730613760 bytes 2026-02-19T20:19:48.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4799021056 bytes 2026-02-19T20:19:48.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4848330752 bytes 2026-02-19T20:19:49.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4899634176 bytes 2026-02-19T20:19:49.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4931469312 bytes 2026-02-19T20:19:49.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4981419008 bytes 2026-02-19T20:19:49.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5000514560 bytes 2026-02-19T20:19:49.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5022440448 bytes 2026-02-19T20:19:49.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5071269888 bytes 2026-02-19T20:19:49.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5164566528 bytes 2026-02-19T20:19:49.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5227753472 bytes 2026-02-19T20:19:49.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5277913088 bytes 2026-02-19T20:19:49.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5318981632 bytes 2026-02-19T20:19:49.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5399389184 bytes 2026-02-19T20:19:49.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5478919168 bytes 2026-02-19T20:19:49.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5506739200 bytes 2026-02-19T20:19:49.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5542318080 bytes 2026-02-19T20:19:50.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5563476992 bytes 2026-02-19T20:19:50.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5589414912 bytes 2026-02-19T20:19:50.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5602262016 bytes 2026-02-19T20:19:50.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5627593728 bytes 2026-02-19T20:19:50.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5681759232 bytes 2026-02-19T20:19:50.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5744627712 bytes 2026-02-19T20:19:50.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5788545024 bytes 2026-02-19T20:19:50.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5807637504 bytes 2026-02-19T20:19:50.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5871989760 bytes 2026-02-19T20:19:50.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5879075840 bytes 2026-02-19T20:19:50.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5892876288 bytes 2026-02-19T20:19:50.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5893483520 bytes 2026-02-19T20:19:50.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5925815296 bytes 2026-02-19T20:19:50.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6034161664 bytes 2026-02-19T20:19:50.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6034759680 bytes 2026-02-19T20:19:50.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6080401408 bytes 2026-02-19T20:19:51.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6146310144 bytes 2026-02-19T20:19:51.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6166532096 bytes 2026-02-19T20:19:51.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6220047360 bytes 2026-02-19T20:19:51.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6237627392 bytes 2026-02-19T20:19:51.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6278842368 bytes 2026-02-19T20:19:51.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6316512256 bytes 2026-02-19T20:19:51.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6342763520 bytes 2026-02-19T20:19:51.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6347980800 bytes 2026-02-19T20:19:51.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6428286976 bytes 2026-02-19T20:19:51.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6448335872 bytes 2026-02-19T20:19:51.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6511784960 bytes 2026-02-19T20:19:51.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6557688832 bytes 2026-02-19T20:19:51.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6624821248 bytes 2026-02-19T20:19:51.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6674876416 bytes 2026-02-19T20:19:51.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6737861632 bytes 2026-02-19T20:19:52.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6820054016 bytes 2026-02-19T20:19:52.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6861197312 bytes 2026-02-19T20:19:52.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6864913408 bytes 2026-02-19T20:19:52.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6924936192 bytes 2026-02-19T20:19:52.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6964773888 bytes 2026-02-19T20:19:52.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6983009280 bytes 2026-02-19T20:19:52.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7035980800 bytes 2026-02-19T20:19:52.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7064164352 bytes 2026-02-19T20:19:52.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7111321600 bytes 2026-02-19T20:19:52.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7137861632 bytes 2026-02-19T20:19:52.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7270514688 bytes 2026-02-19T20:19:52.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7272717312 bytes 2026-02-19T20:19:52.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7273303040 bytes 2026-02-19T20:19:52.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7291342848 bytes 2026-02-19T20:19:52.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7326751744 bytes 2026-02-19T20:19:52.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7398823936 bytes 2026-02-19T20:19:53.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7444566016 bytes 2026-02-19T20:19:53.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7483033600 bytes 2026-02-19T20:19:53.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7510691840 bytes 2026-02-19T20:19:53.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7559823360 bytes 2026-02-19T20:19:53.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7614117888 bytes 2026-02-19T20:19:53.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7626552320 bytes 2026-02-19T20:19:53.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7696858112 bytes 2026-02-19T20:19:53.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7731894272 bytes 2026-02-19T20:19:53.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7798081536 bytes 2026-02-19T20:19:53.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7828063232 bytes 2026-02-19T20:19:53.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7864178688 bytes 2026-02-19T20:19:53.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7894149120 bytes 2026-02-19T20:19:53.762 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7950394368 bytes 2026-02-19T20:19:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8018621440 bytes 2026-02-19T20:19:53.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8044906496 bytes 2026-02-19T20:19:53.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8083799040 bytes 2026-02-19T20:19:54.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8190053376 bytes 2026-02-19T20:19:54.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8211380224 bytes 2026-02-19T20:19:54.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8256121856 bytes 2026-02-19T20:19:54.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8312977408 bytes 2026-02-19T20:19:54.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8346672128 bytes 2026-02-19T20:19:54.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8373666816 bytes 2026-02-19T20:19:54.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8405841920 bytes 2026-02-19T20:19:54.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8458454016 bytes 2026-02-19T20:19:54.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8491048960 bytes 2026-02-19T20:19:54.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:19:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:19:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:19:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:19:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_10 2026-02-19T20:19:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_10 snapshot 128M 2026-02-19T20:19:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:19:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:19:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_10 128M 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:19:54.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_10 2026-02-19T20:19:54.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:19:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:19:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:19:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_10 snapshot 2026-02-19T20:19:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:19:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:19:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_10 snapshot 2026-02-19T20:19:55.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:19:55.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_10 2026-02-19T20:19:55.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_10': 2026-02-19T20:19:55.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:19:55.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:19:55.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 12d5e010eacb 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.12d5e010eacb 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:54 2026 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:54 2026 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:54 2026 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:19:55.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:19:55.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: b4b55785-9125-4cba-be08-1b555be092a3 2026-02-19T20:19:55.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:19:55.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:19:55.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_10 2026-02-19T20:19:55.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:19:55.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:19:55.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:19:55.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:19:55.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=7 2026-02-19T20:19:55.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:19:55.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_10 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:19:55.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 617472 bytes 2026-02-19T20:19:55.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 47353856 bytes 2026-02-19T20:19:55.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 78316544 bytes 2026-02-19T20:19:55.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 108065792 bytes 2026-02-19T20:19:55.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 144395264 bytes 2026-02-19T20:19:55.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 150447104 bytes 2026-02-19T20:19:55.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 163072000 bytes 2026-02-19T20:19:55.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 173932544 bytes 2026-02-19T20:19:56.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 194125824 bytes 2026-02-19T20:19:56.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 211911680 bytes 2026-02-19T20:19:56.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 300214272 bytes 2026-02-19T20:19:56.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 346994688 bytes 2026-02-19T20:19:56.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 347624448 bytes 2026-02-19T20:19:56.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 393334784 bytes 2026-02-19T20:19:56.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 417768448 bytes 2026-02-19T20:19:56.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 456595456 bytes 2026-02-19T20:19:56.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 537594880 bytes 2026-02-19T20:19:56.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 562320384 bytes 2026-02-19T20:19:56.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 593436672 bytes 2026-02-19T20:19:56.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 620552192 bytes 2026-02-19T20:19:56.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 638564352 bytes 2026-02-19T20:19:56.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 684224512 bytes 2026-02-19T20:19:56.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 698590208 bytes 2026-02-19T20:19:56.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 754852864 bytes 2026-02-19T20:19:56.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 796339200 bytes 2026-02-19T20:19:56.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 831767552 bytes 2026-02-19T20:19:57.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 857793536 bytes 2026-02-19T20:19:57.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 878780416 bytes 2026-02-19T20:19:57.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 962994176 bytes 2026-02-19T20:19:57.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1015963648 bytes 2026-02-19T20:19:57.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1049152512 bytes 2026-02-19T20:19:57.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1055397888 bytes 2026-02-19T20:19:57.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1149998080 bytes 2026-02-19T20:19:57.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1208309760 bytes 2026-02-19T20:19:57.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1230573568 bytes 2026-02-19T20:19:57.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1309820928 bytes 2026-02-19T20:19:57.593 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:19:57.590+0000 7fab1dc3d640 -1 reset not still connected to 0xb5829c0 2026-02-19T20:19:57.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1353216000 bytes 2026-02-19T20:19:57.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1396730880 bytes 2026-02-19T20:19:57.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1427894272 bytes 2026-02-19T20:19:57.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1490929664 bytes 2026-02-19T20:19:57.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1544315904 bytes 2026-02-19T20:19:57.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1573463040 bytes 2026-02-19T20:19:58.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1630839808 bytes 2026-02-19T20:19:58.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1691636736 bytes 2026-02-19T20:19:58.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1707013120 bytes 2026-02-19T20:19:58.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1713119232 bytes 2026-02-19T20:19:58.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1788349440 bytes 2026-02-19T20:19:58.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1833448448 bytes 2026-02-19T20:19:58.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1846475776 bytes 2026-02-19T20:19:58.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1908445184 bytes 2026-02-19T20:19:58.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1999802368 bytes 2026-02-19T20:19:58.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2048023552 bytes 2026-02-19T20:19:58.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2106591232 bytes 2026-02-19T20:19:58.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2140997632 bytes 2026-02-19T20:19:58.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2219937792 bytes 2026-02-19T20:19:58.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2287318016 bytes 2026-02-19T20:19:58.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2328649728 bytes 2026-02-19T20:19:58.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2361015296 bytes 2026-02-19T20:19:59.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2414406656 bytes 2026-02-19T20:19:59.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2451853312 bytes 2026-02-19T20:19:59.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2475726848 bytes 2026-02-19T20:19:59.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2476341248 bytes 2026-02-19T20:19:59.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2497643520 bytes 2026-02-19T20:19:59.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2581663744 bytes 2026-02-19T20:19:59.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2627683328 bytes 2026-02-19T20:19:59.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2631598080 bytes 2026-02-19T20:19:59.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2661344256 bytes 2026-02-19T20:19:59.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2676702208 bytes 2026-02-19T20:19:59.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2683853824 bytes 2026-02-19T20:19:59.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2751294464 bytes 2026-02-19T20:19:59.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2826015744 bytes 2026-02-19T20:19:59.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2905866240 bytes 2026-02-19T20:19:59.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2949557248 bytes 2026-02-19T20:19:59.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2988483584 bytes 2026-02-19T20:19:59.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3022648320 bytes 2026-02-19T20:19:59.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3023273984 bytes 2026-02-19T20:19:59.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3073546240 bytes 2026-02-19T20:20:00.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3138361344 bytes 2026-02-19T20:20:00.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3165501440 bytes 2026-02-19T20:20:00.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3224310784 bytes 2026-02-19T20:20:00.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3245945856 bytes 2026-02-19T20:20:00.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3252958208 bytes 2026-02-19T20:20:00.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3342633984 bytes 2026-02-19T20:20:00.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3381133312 bytes 2026-02-19T20:20:00.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3394784256 bytes 2026-02-19T20:20:00.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3413232640 bytes 2026-02-19T20:20:00.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3442906112 bytes 2026-02-19T20:20:00.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3504802816 bytes 2026-02-19T20:20:00.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3555215360 bytes 2026-02-19T20:20:00.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3557254144 bytes 2026-02-19T20:20:00.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3645883392 bytes 2026-02-19T20:20:00.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3661818880 bytes 2026-02-19T20:20:00.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3692668928 bytes 2026-02-19T20:20:00.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3716434944 bytes 2026-02-19T20:20:00.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3754071040 bytes 2026-02-19T20:20:01.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3771308032 bytes 2026-02-19T20:20:01.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3816041472 bytes 2026-02-19T20:20:01.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3891477504 bytes 2026-02-19T20:20:01.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3938329600 bytes 2026-02-19T20:20:01.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 3996020736 bytes 2026-02-19T20:20:01.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4068677632 bytes 2026-02-19T20:20:01.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4124957696 bytes 2026-02-19T20:20:01.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4170812416 bytes 2026-02-19T20:20:01.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4188316672 bytes 2026-02-19T20:20:01.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4203208704 bytes 2026-02-19T20:20:01.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4226026496 bytes 2026-02-19T20:20:01.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4251693056 bytes 2026-02-19T20:20:01.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4271798272 bytes 2026-02-19T20:20:01.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4300665856 bytes 2026-02-19T20:20:01.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4352152576 bytes 2026-02-19T20:20:01.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4415953920 bytes 2026-02-19T20:20:02.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4478042112 bytes 2026-02-19T20:20:02.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4498397184 bytes 2026-02-19T20:20:02.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4527973376 bytes 2026-02-19T20:20:02.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4554833920 bytes 2026-02-19T20:20:02.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4606306304 bytes 2026-02-19T20:20:02.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4669371392 bytes 2026-02-19T20:20:02.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4680435712 bytes 2026-02-19T20:20:02.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4737856512 bytes 2026-02-19T20:20:02.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4798492672 bytes 2026-02-19T20:20:02.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4835082240 bytes 2026-02-19T20:20:02.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4913464320 bytes 2026-02-19T20:20:02.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:20:02.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:20:02.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:20:02.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:02.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:20:02.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_11 2026-02-19T20:20:02.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_11 snapshot 128M 2026-02-19T20:20:02.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:02.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:02.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:02.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:02.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:20:02.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:02.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:02.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:20:02.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:02.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_11 128M 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:20:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_11 2026-02-19T20:20:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:20:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:20:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_11 snapshot 2026-02-19T20:20:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:20:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_11 snapshot 2026-02-19T20:20:03.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:20:03.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_11 2026-02-19T20:20:03.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_11': 2026-02-19T20:20:03.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:20:03.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 12e1d2b2b40 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.12e1d2b2b40 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:02 2026 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:02 2026 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:02 2026 2026-02-19T20:20:03.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:20:03.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:20:03.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: d20ca347-64b1-43f6-9ef4-b6887cc47d4b 2026-02-19T20:20:03.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:20:03.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:20:03.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_11 2026-02-19T20:20:03.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:03.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:03.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:20:03.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:20:03.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=14 2026-02-19T20:20:03.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:20:03.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_11 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:20:03.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 20278272 bytes 2026-02-19T20:20:03.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 60047360 bytes 2026-02-19T20:20:03.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 84230144 bytes 2026-02-19T20:20:03.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 112405504 bytes 2026-02-19T20:20:03.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 166953984 bytes 2026-02-19T20:20:03.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 176122880 bytes 2026-02-19T20:20:03.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 176708608 bytes 2026-02-19T20:20:03.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 187194368 bytes 2026-02-19T20:20:04.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 242496512 bytes 2026-02-19T20:20:04.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 271725568 bytes 2026-02-19T20:20:04.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 326580224 bytes 2026-02-19T20:20:04.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 370340864 bytes 2026-02-19T20:20:04.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 391920640 bytes 2026-02-19T20:20:04.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 427912192 bytes 2026-02-19T20:20:04.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 472637440 bytes 2026-02-19T20:20:04.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 505203712 bytes 2026-02-19T20:20:04.474 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2026-02-19T20:20:04.474 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2026-02-19T20:20:04.474 INFO:tasks.rbd_mirror_thrash:waiting for 5.795468566226213 secs before reviving daemons 2026-02-19T20:20:04.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:20:04.473+0000 7fbebda8e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53027) UID: 1000 2026-02-19T20:20:04.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 516686848 bytes 2026-02-19T20:20:04.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 564537344 bytes 2026-02-19T20:20:04.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 600281088 bytes 2026-02-19T20:20:04.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 666456064 bytes 2026-02-19T20:20:04.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 709504000 bytes 2026-02-19T20:20:04.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 733726720 bytes 2026-02-19T20:20:04.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 786279424 bytes 2026-02-19T20:20:04.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 850661376 bytes 2026-02-19T20:20:05.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 947735552 bytes 2026-02-19T20:20:05.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1034063872 bytes 2026-02-19T20:20:05.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1087173632 bytes 2026-02-19T20:20:05.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1127866368 bytes 2026-02-19T20:20:05.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1148429312 bytes 2026-02-19T20:20:05.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1179986944 bytes 2026-02-19T20:20:05.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1279474688 bytes 2026-02-19T20:20:05.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1293441024 bytes 2026-02-19T20:20:05.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1342904320 bytes 2026-02-19T20:20:05.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1407477760 bytes 2026-02-19T20:20:05.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1413344256 bytes 2026-02-19T20:20:05.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1447201792 bytes 2026-02-19T20:20:05.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1505549312 bytes 2026-02-19T20:20:05.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1520133120 bytes 2026-02-19T20:20:05.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1537840128 bytes 2026-02-19T20:20:05.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1548900352 bytes 2026-02-19T20:20:05.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1586350080 bytes 2026-02-19T20:20:06.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1620657152 bytes 2026-02-19T20:20:06.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1665426432 bytes 2026-02-19T20:20:06.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1714061312 bytes 2026-02-19T20:20:06.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1723188224 bytes 2026-02-19T20:20:06.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1759618048 bytes 2026-02-19T20:20:06.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1821875200 bytes 2026-02-19T20:20:06.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1847570432 bytes 2026-02-19T20:20:06.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1848190976 bytes 2026-02-19T20:20:06.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1893997568 bytes 2026-02-19T20:20:06.530 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4393223775040953) > weight (0.25) 2026-02-19T20:20:06.530 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4883974762795604) > weight (0.25) 2026-02-19T20:20:06.530 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9207433087012216) > weight (0.25) 2026-02-19T20:20:06.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.429588868841023) > weight (0.25) 2026-02-19T20:20:06.531 INFO:tasks.rbd_mirror_thrash:waiting for 105.52726689094504 secs before thrashing 2026-02-19T20:20:06.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1968635904 bytes 2026-02-19T20:20:06.600 INFO:tasks.ceph.cluster2.osd.1.trial080.stderr:2026-02-19T20:20:06.597+0000 7f1a949a9640 -1 reset not still connected to 0x4681860 2026-02-19T20:20:06.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2002778112 bytes 2026-02-19T20:20:06.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2084648960 bytes 2026-02-19T20:20:06.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2104968192 bytes 2026-02-19T20:20:06.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2141452288 bytes 2026-02-19T20:20:06.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2183661568 bytes 2026-02-19T20:20:06.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2245617664 bytes 2026-02-19T20:20:06.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2266182656 bytes 2026-02-19T20:20:07.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2321439744 bytes 2026-02-19T20:20:07.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2371137536 bytes 2026-02-19T20:20:07.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2433624064 bytes 2026-02-19T20:20:07.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2482719744 bytes 2026-02-19T20:20:07.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2483338240 bytes 2026-02-19T20:20:07.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2530322432 bytes 2026-02-19T20:20:07.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2530964480 bytes 2026-02-19T20:20:07.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2597106688 bytes 2026-02-19T20:20:07.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2644066304 bytes 2026-02-19T20:20:07.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2672642048 bytes 2026-02-19T20:20:07.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2679638016 bytes 2026-02-19T20:20:07.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2709376000 bytes 2026-02-19T20:20:07.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2795301888 bytes 2026-02-19T20:20:07.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2798103552 bytes 2026-02-19T20:20:07.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2820473856 bytes 2026-02-19T20:20:07.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2823975936 bytes 2026-02-19T20:20:07.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2843129856 bytes 2026-02-19T20:20:07.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2864958464 bytes 2026-02-19T20:20:07.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 2921634816 bytes 2026-02-19T20:20:08.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 2968802304 bytes 2026-02-19T20:20:08.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 2969404416 bytes 2026-02-19T20:20:08.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 2978778112 bytes 2026-02-19T20:20:08.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 2979403776 bytes 2026-02-19T20:20:08.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3039243264 bytes 2026-02-19T20:20:08.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3059032064 bytes 2026-02-19T20:20:08.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3078341632 bytes 2026-02-19T20:20:08.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3174299648 bytes 2026-02-19T20:20:08.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3211025408 bytes 2026-02-19T20:20:08.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3240588288 bytes 2026-02-19T20:20:08.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3259107328 bytes 2026-02-19T20:20:08.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3261654016 bytes 2026-02-19T20:20:08.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3315733504 bytes 2026-02-19T20:20:08.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3381098496 bytes 2026-02-19T20:20:08.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3438708736 bytes 2026-02-19T20:20:09.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3501223936 bytes 2026-02-19T20:20:09.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3550074880 bytes 2026-02-19T20:20:09.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3668877312 bytes 2026-02-19T20:20:09.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3685598208 bytes 2026-02-19T20:20:09.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3735528448 bytes 2026-02-19T20:20:09.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3774782464 bytes 2026-02-19T20:20:09.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3792888832 bytes 2026-02-19T20:20:09.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 3844946944 bytes 2026-02-19T20:20:09.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 3928568832 bytes 2026-02-19T20:20:09.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 3957898240 bytes 2026-02-19T20:20:09.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 3960254464 bytes 2026-02-19T20:20:09.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 3969213440 bytes 2026-02-19T20:20:09.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4027891712 bytes 2026-02-19T20:20:10.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4074529792 bytes 2026-02-19T20:20:10.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4102373376 bytes 2026-02-19T20:20:10.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4206615552 bytes 2026-02-19T20:20:10.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4216523776 bytes 2026-02-19T20:20:10.271 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2026-02-19T20:20:10.271 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:20:10.272 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2026-02-19T20:20:10.272 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2026-02-19T20:20:10.272 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2026-02-19T20:20:10.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2026-02-19T20:20:10.275 INFO:tasks.rbd_mirror_thrash:waiting for 104.68441210923271 secs before thrashing 2026-02-19T20:20:10.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4245927936 bytes 2026-02-19T20:20:10.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4263636992 bytes 2026-02-19T20:20:10.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4298614784 bytes 2026-02-19T20:20:10.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4327645184 bytes 2026-02-19T20:20:10.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4369095680 bytes 2026-02-19T20:20:10.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4385440768 bytes 2026-02-19T20:20:10.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4427283456 bytes 2026-02-19T20:20:10.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4473425920 bytes 2026-02-19T20:20:10.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4518590464 bytes 2026-02-19T20:20:10.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4610842624 bytes 2026-02-19T20:20:10.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4685757440 bytes 2026-02-19T20:20:11.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4686363648 bytes 2026-02-19T20:20:11.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4717671424 bytes 2026-02-19T20:20:11.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4738417664 bytes 2026-02-19T20:20:11.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4744684544 bytes 2026-02-19T20:20:11.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 4783007744 bytes 2026-02-19T20:20:11.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 4828270592 bytes 2026-02-19T20:20:11.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 4844276736 bytes 2026-02-19T20:20:11.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 4867540992 bytes 2026-02-19T20:20:11.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 4879515648 bytes 2026-02-19T20:20:11.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 4916174848 bytes 2026-02-19T20:20:11.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 4957272064 bytes 2026-02-19T20:20:11.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 4998806528 bytes 2026-02-19T20:20:11.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5069579264 bytes 2026-02-19T20:20:11.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5140772864 bytes 2026-02-19T20:20:11.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5226976256 bytes 2026-02-19T20:20:11.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5274635264 bytes 2026-02-19T20:20:11.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5367212032 bytes 2026-02-19T20:20:12.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5387274240 bytes 2026-02-19T20:20:12.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5452505088 bytes 2026-02-19T20:20:12.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5464898560 bytes 2026-02-19T20:20:12.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5484052480 bytes 2026-02-19T20:20:12.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5560074240 bytes 2026-02-19T20:20:12.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5617768448 bytes 2026-02-19T20:20:12.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5702385664 bytes 2026-02-19T20:20:12.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5741012992 bytes 2026-02-19T20:20:12.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5743666176 bytes 2026-02-19T20:20:12.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5850315776 bytes 2026-02-19T20:20:12.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5901679616 bytes 2026-02-19T20:20:12.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 5902301184 bytes 2026-02-19T20:20:12.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 5905108992 bytes 2026-02-19T20:20:12.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 5921965056 bytes 2026-02-19T20:20:12.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 5970949120 bytes 2026-02-19T20:20:12.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6016419840 bytes 2026-02-19T20:20:12.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6072825856 bytes 2026-02-19T20:20:13.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6110113792 bytes 2026-02-19T20:20:13.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6163155968 bytes 2026-02-19T20:20:13.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6175661056 bytes 2026-02-19T20:20:13.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6192833536 bytes 2026-02-19T20:20:13.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6240388096 bytes 2026-02-19T20:20:13.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6285677568 bytes 2026-02-19T20:20:13.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6328286208 bytes 2026-02-19T20:20:13.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6328915968 bytes 2026-02-19T20:20:13.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6365065216 bytes 2026-02-19T20:20:13.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6443022336 bytes 2026-02-19T20:20:13.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6460874752 bytes 2026-02-19T20:20:13.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6526245888 bytes 2026-02-19T20:20:13.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6594015232 bytes 2026-02-19T20:20:13.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6647348224 bytes 2026-02-19T20:20:13.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6700403712 bytes 2026-02-19T20:20:13.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6768520192 bytes 2026-02-19T20:20:14.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6795976704 bytes 2026-02-19T20:20:14.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 6903883776 bytes 2026-02-19T20:20:14.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 6974891008 bytes 2026-02-19T20:20:14.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 6987197440 bytes 2026-02-19T20:20:14.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7011786752 bytes 2026-02-19T20:20:14.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7085088768 bytes 2026-02-19T20:20:14.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7154637824 bytes 2026-02-19T20:20:14.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7198377984 bytes 2026-02-19T20:20:14.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7249190912 bytes 2026-02-19T20:20:14.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7293359104 bytes 2026-02-19T20:20:14.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7340443648 bytes 2026-02-19T20:20:14.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7381318656 bytes 2026-02-19T20:20:14.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7421450240 bytes 2026-02-19T20:20:14.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7476768768 bytes 2026-02-19T20:20:14.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7503048704 bytes 2026-02-19T20:20:15.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7560204288 bytes 2026-02-19T20:20:15.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7574218752 bytes 2026-02-19T20:20:15.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7601196032 bytes 2026-02-19T20:20:15.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7604334592 bytes 2026-02-19T20:20:15.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7656731648 bytes 2026-02-19T20:20:15.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7711407104 bytes 2026-02-19T20:20:15.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7747895296 bytes 2026-02-19T20:20:15.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7749938176 bytes 2026-02-19T20:20:15.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 7791849472 bytes 2026-02-19T20:20:15.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 7835997184 bytes 2026-02-19T20:20:15.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 7852789760 bytes 2026-02-19T20:20:15.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 7882689536 bytes 2026-02-19T20:20:15.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 7916852224 bytes 2026-02-19T20:20:15.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 7975925760 bytes 2026-02-19T20:20:15.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8013262848 bytes 2026-02-19T20:20:15.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8019628032 bytes 2026-02-19T20:20:15.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8070786048 bytes 2026-02-19T20:20:15.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8095774720 bytes 2026-02-19T20:20:15.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8108931072 bytes 2026-02-19T20:20:15.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8140526592 bytes 2026-02-19T20:20:16.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8183974912 bytes 2026-02-19T20:20:16.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8233904128 bytes 2026-02-19T20:20:16.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8276595712 bytes 2026-02-19T20:20:16.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 8311005184 bytes 2026-02-19T20:20:16.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 8373294080 bytes 2026-02-19T20:20:16.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 8429143040 bytes 2026-02-19T20:20:16.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 8481220608 bytes 2026-02-19T20:20:16.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 8548477952 bytes 2026-02-19T20:20:16.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 8561498112 bytes 2026-02-19T20:20:16.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 8604516352 bytes 2026-02-19T20:20:16.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 8647785472 bytes 2026-02-19T20:20:16.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 8648691712 bytes 2026-02-19T20:20:16.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 8745195520 bytes 2026-02-19T20:20:16.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 8745838592 bytes 2026-02-19T20:20:16.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 8777443328 bytes 2026-02-19T20:20:16.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 8807132160 bytes 2026-02-19T20:20:17.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 8883686400 bytes 2026-02-19T20:20:17.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 8939904000 bytes 2026-02-19T20:20:17.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 8952588288 bytes 2026-02-19T20:20:17.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 9041737728 bytes 2026-02-19T20:20:17.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22600 IOs, 9105415168 bytes 2026-02-19T20:20:17.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22700 IOs, 9132904448 bytes 2026-02-19T20:20:17.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22800 IOs, 9157321728 bytes 2026-02-19T20:20:17.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22900 IOs, 9223767040 bytes 2026-02-19T20:20:17.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23000 IOs, 9287328768 bytes 2026-02-19T20:20:17.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:20:17.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:20:17.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:20:17.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:17.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:20:17.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_12 2026-02-19T20:20:17.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_12 snapshot 128M 2026-02-19T20:20:17.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:17.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_12 128M 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:17.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:20:17.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:17.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:20:17.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:20:17.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:20:17.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:17.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_12 2026-02-19T20:20:17.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:20:17.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:17.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:20:17.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:17.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_12 snapshot 2026-02-19T20:20:17.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:17.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:17.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:20:17.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:17.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_12 snapshot 2026-02-19T20:20:18.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:20:18.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_12 2026-02-19T20:20:18.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_12': 2026-02-19T20:20:18.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:20:18.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:20:18.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:20:18.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 12f511da3043 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.12f511da3043 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:17 2026 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:17 2026 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:17 2026 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:20:18.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:20:18.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: a442b5dd-49c6-40c0-a740-73e9245e620d 2026-02-19T20:20:18.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:20:18.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:20:18.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_12 2026-02-19T20:20:18.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:18.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:18.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:20:18.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:20:18.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=8 2026-02-19T20:20:18.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:20:18.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_12 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:20:18.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 35359744 bytes 2026-02-19T20:20:18.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 128922624 bytes 2026-02-19T20:20:18.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 171003904 bytes 2026-02-19T20:20:18.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 222645248 bytes 2026-02-19T20:20:18.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 321360896 bytes 2026-02-19T20:20:19.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 371162112 bytes 2026-02-19T20:20:19.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 419169280 bytes 2026-02-19T20:20:19.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 458872832 bytes 2026-02-19T20:20:19.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 563274752 bytes 2026-02-19T20:20:19.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 633084928 bytes 2026-02-19T20:20:19.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 698645504 bytes 2026-02-19T20:20:19.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 756924416 bytes 2026-02-19T20:20:19.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 769244160 bytes 2026-02-19T20:20:19.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 792103936 bytes 2026-02-19T20:20:19.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 799952896 bytes 2026-02-19T20:20:19.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 867198976 bytes 2026-02-19T20:20:19.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 911155200 bytes 2026-02-19T20:20:19.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 926874624 bytes 2026-02-19T20:20:19.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 955724800 bytes 2026-02-19T20:20:19.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 986025984 bytes 2026-02-19T20:20:19.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1019382784 bytes 2026-02-19T20:20:20.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1086735360 bytes 2026-02-19T20:20:20.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1127291904 bytes 2026-02-19T20:20:20.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1181511680 bytes 2026-02-19T20:20:20.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1259056128 bytes 2026-02-19T20:20:20.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1305487360 bytes 2026-02-19T20:20:20.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1347013632 bytes 2026-02-19T20:20:20.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1400758272 bytes 2026-02-19T20:20:20.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1440512000 bytes 2026-02-19T20:20:20.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1490448384 bytes 2026-02-19T20:20:20.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1532497920 bytes 2026-02-19T20:20:20.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1571953664 bytes 2026-02-19T20:20:20.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1626547200 bytes 2026-02-19T20:20:20.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1694213120 bytes 2026-02-19T20:20:20.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1695819776 bytes 2026-02-19T20:20:20.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1727695872 bytes 2026-02-19T20:20:21.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1800630272 bytes 2026-02-19T20:20:21.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1806405632 bytes 2026-02-19T20:20:21.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1825738752 bytes 2026-02-19T20:20:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1850940416 bytes 2026-02-19T20:20:21.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1905511424 bytes 2026-02-19T20:20:21.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1912914944 bytes 2026-02-19T20:20:21.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1953642496 bytes 2026-02-19T20:20:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1982957568 bytes 2026-02-19T20:20:21.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2001524736 bytes 2026-02-19T20:20:21.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2038687744 bytes 2026-02-19T20:20:21.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2061869056 bytes 2026-02-19T20:20:21.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2062479360 bytes 2026-02-19T20:20:21.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2113736704 bytes 2026-02-19T20:20:21.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2141146112 bytes 2026-02-19T20:20:21.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2174012416 bytes 2026-02-19T20:20:21.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2262293504 bytes 2026-02-19T20:20:22.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2270108672 bytes 2026-02-19T20:20:22.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2334144512 bytes 2026-02-19T20:20:22.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2347432960 bytes 2026-02-19T20:20:22.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2393798656 bytes 2026-02-19T20:20:22.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2448300032 bytes 2026-02-19T20:20:22.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2453609472 bytes 2026-02-19T20:20:22.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2515591168 bytes 2026-02-19T20:20:22.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2519292928 bytes 2026-02-19T20:20:22.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2584665088 bytes 2026-02-19T20:20:22.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2606858240 bytes 2026-02-19T20:20:22.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2629967872 bytes 2026-02-19T20:20:22.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2631809024 bytes 2026-02-19T20:20:22.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2686853120 bytes 2026-02-19T20:20:22.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2726473728 bytes 2026-02-19T20:20:22.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2792778752 bytes 2026-02-19T20:20:22.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2833425408 bytes 2026-02-19T20:20:22.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2848457728 bytes 2026-02-19T20:20:22.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2896794624 bytes 2026-02-19T20:20:23.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2918705152 bytes 2026-02-19T20:20:23.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2927912960 bytes 2026-02-19T20:20:23.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2928552960 bytes 2026-02-19T20:20:23.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2994611200 bytes 2026-02-19T20:20:23.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3019141120 bytes 2026-02-19T20:20:23.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3024707584 bytes 2026-02-19T20:20:23.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3025337344 bytes 2026-02-19T20:20:23.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3054465024 bytes 2026-02-19T20:20:23.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3102530560 bytes 2026-02-19T20:20:23.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3139776512 bytes 2026-02-19T20:20:23.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3165005824 bytes 2026-02-19T20:20:23.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3197038592 bytes 2026-02-19T20:20:23.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3238403072 bytes 2026-02-19T20:20:23.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3307670528 bytes 2026-02-19T20:20:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3334550528 bytes 2026-02-19T20:20:23.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3416677376 bytes 2026-02-19T20:20:23.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3436069888 bytes 2026-02-19T20:20:23.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3485783040 bytes 2026-02-19T20:20:23.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3527012352 bytes 2026-02-19T20:20:24.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3557096448 bytes 2026-02-19T20:20:24.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3621972992 bytes 2026-02-19T20:20:24.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3715719168 bytes 2026-02-19T20:20:24.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3773289472 bytes 2026-02-19T20:20:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3798661120 bytes 2026-02-19T20:20:24.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3799249920 bytes 2026-02-19T20:20:24.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3813835776 bytes 2026-02-19T20:20:24.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3849682944 bytes 2026-02-19T20:20:24.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3870492672 bytes 2026-02-19T20:20:24.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3903401984 bytes 2026-02-19T20:20:24.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 3965845504 bytes 2026-02-19T20:20:24.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 3966448640 bytes 2026-02-19T20:20:24.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 3967058944 bytes 2026-02-19T20:20:24.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 3983161344 bytes 2026-02-19T20:20:24.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4050781184 bytes 2026-02-19T20:20:24.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4063421440 bytes 2026-02-19T20:20:24.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4075739136 bytes 2026-02-19T20:20:24.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4093372416 bytes 2026-02-19T20:20:24.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4139602944 bytes 2026-02-19T20:20:25.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4164331520 bytes 2026-02-19T20:20:25.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4233453568 bytes 2026-02-19T20:20:25.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4242238464 bytes 2026-02-19T20:20:25.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4297159680 bytes 2026-02-19T20:20:25.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4321478656 bytes 2026-02-19T20:20:25.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4353693696 bytes 2026-02-19T20:20:25.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4409040896 bytes 2026-02-19T20:20:25.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4466200576 bytes 2026-02-19T20:20:25.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4541954048 bytes 2026-02-19T20:20:25.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4589851648 bytes 2026-02-19T20:20:25.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4625819648 bytes 2026-02-19T20:20:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4699589632 bytes 2026-02-19T20:20:25.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4700199936 bytes 2026-02-19T20:20:25.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4738145280 bytes 2026-02-19T20:20:25.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4760328192 bytes 2026-02-19T20:20:26.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4817204224 bytes 2026-02-19T20:20:26.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 4855240704 bytes 2026-02-19T20:20:26.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 4871298048 bytes 2026-02-19T20:20:26.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 4919410688 bytes 2026-02-19T20:20:26.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 4933547008 bytes 2026-02-19T20:20:26.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5031847936 bytes 2026-02-19T20:20:26.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5039442944 bytes 2026-02-19T20:20:26.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5127725056 bytes 2026-02-19T20:20:26.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5159496704 bytes 2026-02-19T20:20:26.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:20:26.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_13 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_13 snapshot 128M 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:20:26.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_13 128M 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:20:26.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:20:26.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:20:26.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:26.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_13 2026-02-19T20:20:26.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:20:26.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:26.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:20:26.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:26.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_13 snapshot 2026-02-19T20:20:26.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:26.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:26.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:20:26.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:26.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_13 snapshot 2026-02-19T20:20:27.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:20:27.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_13 2026-02-19T20:20:27.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_13': 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 130125eae6e7 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.130125eae6e7 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:26 2026 2026-02-19T20:20:27.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:26 2026 2026-02-19T20:20:27.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:26 2026 2026-02-19T20:20:27.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:20:27.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:20:27.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: b748d40b-d69e-4188-bd35-09d03d9a991d 2026-02-19T20:20:27.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:20:27.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:20:27.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_13 2026-02-19T20:20:27.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:27.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:27.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:20:27.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:20:27.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=14 2026-02-19T20:20:27.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:20:27.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_13 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:20:27.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 38293504 bytes 2026-02-19T20:20:27.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 78255104 bytes 2026-02-19T20:20:27.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 162979840 bytes 2026-02-19T20:20:27.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 234054656 bytes 2026-02-19T20:20:27.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 267431936 bytes 2026-02-19T20:20:28.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 314440704 bytes 2026-02-19T20:20:28.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 345508864 bytes 2026-02-19T20:20:28.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 370978816 bytes 2026-02-19T20:20:28.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 410770432 bytes 2026-02-19T20:20:28.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 491180032 bytes 2026-02-19T20:20:28.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 535493632 bytes 2026-02-19T20:20:28.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 575692800 bytes 2026-02-19T20:20:28.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 602464256 bytes 2026-02-19T20:20:28.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 625853440 bytes 2026-02-19T20:20:28.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 664868864 bytes 2026-02-19T20:20:28.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 731772928 bytes 2026-02-19T20:20:28.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 800536576 bytes 2026-02-19T20:20:28.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 825250816 bytes 2026-02-19T20:20:28.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 907520000 bytes 2026-02-19T20:20:28.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 965598208 bytes 2026-02-19T20:20:28.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1022018560 bytes 2026-02-19T20:20:28.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1052225536 bytes 2026-02-19T20:20:29.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1056351232 bytes 2026-02-19T20:20:29.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1081119744 bytes 2026-02-19T20:20:29.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1113842688 bytes 2026-02-19T20:20:29.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1146206208 bytes 2026-02-19T20:20:29.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1182508032 bytes 2026-02-19T20:20:29.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1216274432 bytes 2026-02-19T20:20:29.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1239939072 bytes 2026-02-19T20:20:29.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1281868800 bytes 2026-02-19T20:20:29.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1301403648 bytes 2026-02-19T20:20:29.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1368631296 bytes 2026-02-19T20:20:29.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1375709184 bytes 2026-02-19T20:20:29.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1425464320 bytes 2026-02-19T20:20:29.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1511068672 bytes 2026-02-19T20:20:29.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1542380544 bytes 2026-02-19T20:20:29.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1601494016 bytes 2026-02-19T20:20:29.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1607884800 bytes 2026-02-19T20:20:30.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1628642304 bytes 2026-02-19T20:20:30.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1629258752 bytes 2026-02-19T20:20:30.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1658373120 bytes 2026-02-19T20:20:30.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1700226048 bytes 2026-02-19T20:20:30.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1749047296 bytes 2026-02-19T20:20:30.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1775932416 bytes 2026-02-19T20:20:30.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1782575104 bytes 2026-02-19T20:20:30.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1825179648 bytes 2026-02-19T20:20:30.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1825794048 bytes 2026-02-19T20:20:30.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1885096960 bytes 2026-02-19T20:20:30.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1933944832 bytes 2026-02-19T20:20:30.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1941839872 bytes 2026-02-19T20:20:30.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2003786752 bytes 2026-02-19T20:20:30.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2038403072 bytes 2026-02-19T20:20:30.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2046973952 bytes 2026-02-19T20:20:30.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2095208448 bytes 2026-02-19T20:20:30.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2109830144 bytes 2026-02-19T20:20:30.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2115976192 bytes 2026-02-19T20:20:30.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2166520832 bytes 2026-02-19T20:20:30.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2184535040 bytes 2026-02-19T20:20:31.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2242667520 bytes 2026-02-19T20:20:31.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2298536960 bytes 2026-02-19T20:20:31.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2341645312 bytes 2026-02-19T20:20:31.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2387899392 bytes 2026-02-19T20:20:31.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2439308288 bytes 2026-02-19T20:20:31.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2512139264 bytes 2026-02-19T20:20:31.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2537234432 bytes 2026-02-19T20:20:31.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2537840640 bytes 2026-02-19T20:20:31.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2557132800 bytes 2026-02-19T20:20:31.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2577004544 bytes 2026-02-19T20:20:31.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2634511360 bytes 2026-02-19T20:20:31.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2646184960 bytes 2026-02-19T20:20:31.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2654873600 bytes 2026-02-19T20:20:31.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2674210816 bytes 2026-02-19T20:20:31.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2691748864 bytes 2026-02-19T20:20:31.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2791775232 bytes 2026-02-19T20:20:31.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2832742400 bytes 2026-02-19T20:20:32.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2909239296 bytes 2026-02-19T20:20:32.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 2980715520 bytes 2026-02-19T20:20:32.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3022794752 bytes 2026-02-19T20:20:32.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3069582336 bytes 2026-02-19T20:20:32.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3116260352 bytes 2026-02-19T20:20:32.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3151532032 bytes 2026-02-19T20:20:32.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3213422592 bytes 2026-02-19T20:20:32.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3256842240 bytes 2026-02-19T20:20:32.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3295089664 bytes 2026-02-19T20:20:32.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3338307584 bytes 2026-02-19T20:20:32.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3358324736 bytes 2026-02-19T20:20:32.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3380646912 bytes 2026-02-19T20:20:32.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3405120512 bytes 2026-02-19T20:20:32.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3499594752 bytes 2026-02-19T20:20:32.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3565445120 bytes 2026-02-19T20:20:32.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3581280256 bytes 2026-02-19T20:20:33.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3674289152 bytes 2026-02-19T20:20:33.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3767577600 bytes 2026-02-19T20:20:33.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3821930496 bytes 2026-02-19T20:20:33.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3853025280 bytes 2026-02-19T20:20:33.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3870589952 bytes 2026-02-19T20:20:33.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3922578432 bytes 2026-02-19T20:20:33.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3944605696 bytes 2026-02-19T20:20:33.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3964598272 bytes 2026-02-19T20:20:33.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 3980151808 bytes 2026-02-19T20:20:33.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4062200832 bytes 2026-02-19T20:20:33.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4065187840 bytes 2026-02-19T20:20:33.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4121427968 bytes 2026-02-19T20:20:33.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4131208192 bytes 2026-02-19T20:20:33.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4131828736 bytes 2026-02-19T20:20:34.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4186254336 bytes 2026-02-19T20:20:34.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4260311040 bytes 2026-02-19T20:20:34.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4358321152 bytes 2026-02-19T20:20:34.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4375346176 bytes 2026-02-19T20:20:34.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4400719872 bytes 2026-02-19T20:20:34.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4436147200 bytes 2026-02-19T20:20:34.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4458327040 bytes 2026-02-19T20:20:34.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4549103616 bytes 2026-02-19T20:20:34.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4615310336 bytes 2026-02-19T20:20:34.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4659838976 bytes 2026-02-19T20:20:34.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4707552256 bytes 2026-02-19T20:20:34.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4749359104 bytes 2026-02-19T20:20:34.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4765163520 bytes 2026-02-19T20:20:34.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4810518528 bytes 2026-02-19T20:20:35.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4861081600 bytes 2026-02-19T20:20:35.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4872465408 bytes 2026-02-19T20:20:35.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4928954368 bytes 2026-02-19T20:20:35.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4981151744 bytes 2026-02-19T20:20:35.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5031764992 bytes 2026-02-19T20:20:35.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5061756928 bytes 2026-02-19T20:20:35.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5172262912 bytes 2026-02-19T20:20:35.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5210335232 bytes 2026-02-19T20:20:35.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5249676288 bytes 2026-02-19T20:20:35.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5318658048 bytes 2026-02-19T20:20:35.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5357356032 bytes 2026-02-19T20:20:35.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5444051968 bytes 2026-02-19T20:20:35.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5444631552 bytes 2026-02-19T20:20:35.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5481474048 bytes 2026-02-19T20:20:35.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5534660608 bytes 2026-02-19T20:20:35.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5541975040 bytes 2026-02-19T20:20:35.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5564352512 bytes 2026-02-19T20:20:36.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5595672576 bytes 2026-02-19T20:20:36.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5667733504 bytes 2026-02-19T20:20:36.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5680064512 bytes 2026-02-19T20:20:36.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5742989312 bytes 2026-02-19T20:20:36.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5762921472 bytes 2026-02-19T20:20:36.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5785610240 bytes 2026-02-19T20:20:36.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5815897088 bytes 2026-02-19T20:20:36.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5856241664 bytes 2026-02-19T20:20:36.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5934870528 bytes 2026-02-19T20:20:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5992732672 bytes 2026-02-19T20:20:36.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6045164544 bytes 2026-02-19T20:20:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6110014464 bytes 2026-02-19T20:20:36.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6152855552 bytes 2026-02-19T20:20:36.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6156918784 bytes 2026-02-19T20:20:36.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6224122880 bytes 2026-02-19T20:20:36.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6247956480 bytes 2026-02-19T20:20:36.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6270248960 bytes 2026-02-19T20:20:37.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6273591296 bytes 2026-02-19T20:20:37.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6312485888 bytes 2026-02-19T20:20:37.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6362582016 bytes 2026-02-19T20:20:37.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6369793024 bytes 2026-02-19T20:20:37.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6370397184 bytes 2026-02-19T20:20:37.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6465297408 bytes 2026-02-19T20:20:37.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6515722240 bytes 2026-02-19T20:20:37.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6587947008 bytes 2026-02-19T20:20:37.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6604878848 bytes 2026-02-19T20:20:37.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6621739008 bytes 2026-02-19T20:20:37.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6636243968 bytes 2026-02-19T20:20:37.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6642445312 bytes 2026-02-19T20:20:37.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6672387072 bytes 2026-02-19T20:20:37.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6696620032 bytes 2026-02-19T20:20:37.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6801936384 bytes 2026-02-19T20:20:37.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6848575488 bytes 2026-02-19T20:20:37.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6852641792 bytes 2026-02-19T20:20:37.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6955010048 bytes 2026-02-19T20:20:38.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7019039744 bytes 2026-02-19T20:20:38.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7133302784 bytes 2026-02-19T20:20:38.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7240040448 bytes 2026-02-19T20:20:38.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7262130176 bytes 2026-02-19T20:20:38.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7285468160 bytes 2026-02-19T20:20:38.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7350800384 bytes 2026-02-19T20:20:38.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7399450624 bytes 2026-02-19T20:20:38.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7436908544 bytes 2026-02-19T20:20:38.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7475109888 bytes 2026-02-19T20:20:38.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7509211136 bytes 2026-02-19T20:20:38.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7553615872 bytes 2026-02-19T20:20:38.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7580906496 bytes 2026-02-19T20:20:38.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7625481216 bytes 2026-02-19T20:20:38.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7687283712 bytes 2026-02-19T20:20:38.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7726343168 bytes 2026-02-19T20:20:39.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7796170752 bytes 2026-02-19T20:20:39.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7796787200 bytes 2026-02-19T20:20:39.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7797394432 bytes 2026-02-19T20:20:39.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7860713472 bytes 2026-02-19T20:20:39.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7869798400 bytes 2026-02-19T20:20:39.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7911314432 bytes 2026-02-19T20:20:39.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7919178752 bytes 2026-02-19T20:20:39.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 7960270848 bytes 2026-02-19T20:20:39.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 7973146624 bytes 2026-02-19T20:20:39.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 7981708288 bytes 2026-02-19T20:20:39.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 7991332864 bytes 2026-02-19T20:20:39.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8039817216 bytes 2026-02-19T20:20:39.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8120961024 bytes 2026-02-19T20:20:39.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8157388800 bytes 2026-02-19T20:20:39.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8175645696 bytes 2026-02-19T20:20:39.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8247300096 bytes 2026-02-19T20:20:39.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8280444928 bytes 2026-02-19T20:20:39.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8306153472 bytes 2026-02-19T20:20:39.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8390823936 bytes 2026-02-19T20:20:40.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8405356544 bytes 2026-02-19T20:20:40.056 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8420822016 bytes 2026-02-19T20:20:40.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8463974400 bytes 2026-02-19T20:20:40.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 8488254464 bytes 2026-02-19T20:20:40.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 8500567040 bytes 2026-02-19T20:20:40.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 8533667840 bytes 2026-02-19T20:20:40.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 8556555264 bytes 2026-02-19T20:20:40.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 8604556288 bytes 2026-02-19T20:20:40.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 8628542464 bytes 2026-02-19T20:20:40.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 8671738880 bytes 2026-02-19T20:20:40.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 8679429120 bytes 2026-02-19T20:20:40.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 8730480640 bytes 2026-02-19T20:20:40.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 8767634432 bytes 2026-02-19T20:20:40.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 8801554432 bytes 2026-02-19T20:20:40.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 8855696384 bytes 2026-02-19T20:20:40.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 8939025408 bytes 2026-02-19T20:20:40.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 8946274304 bytes 2026-02-19T20:20:40.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 9011383296 bytes 2026-02-19T20:20:41.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 9102879744 bytes 2026-02-19T20:20:41.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 9129624576 bytes 2026-02-19T20:20:41.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22600 IOs, 9162738688 bytes 2026-02-19T20:20:41.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22700 IOs, 9217146880 bytes 2026-02-19T20:20:41.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22800 IOs, 9266996224 bytes 2026-02-19T20:20:41.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22900 IOs, 9327503360 bytes 2026-02-19T20:20:41.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23000 IOs, 9370446848 bytes 2026-02-19T20:20:41.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23100 IOs, 9388115968 bytes 2026-02-19T20:20:41.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23200 IOs, 9423918080 bytes 2026-02-19T20:20:41.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_14 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_14 snapshot 128M 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_14 128M 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:41.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:20:41.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:41.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:20:41.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:20:41.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:20:41.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:41.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_14 2026-02-19T20:20:41.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:20:41.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:41.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:20:41.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:41.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_14 snapshot 2026-02-19T20:20:41.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:20:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_14 snapshot 2026-02-19T20:20:42.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:20:42.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_14 2026-02-19T20:20:42.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_14': 2026-02-19T20:20:42.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:20:42.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:20:42.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:20:42.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 130d42e23078 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.130d42e23078 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:41 2026 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:41 2026 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:41 2026 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:20:42.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:20:42.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 47d7aec0-eb62-49a0-9f34-a0c1ec276d42 2026-02-19T20:20:42.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:20:42.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:20:42.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_14 2026-02-19T20:20:42.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:42.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:42.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:20:42.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:20:42.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=8 2026-02-19T20:20:42.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:20:42.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_14 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:20:42.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 47680512 bytes 2026-02-19T20:20:42.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 86586368 bytes 2026-02-19T20:20:42.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 132675584 bytes 2026-02-19T20:20:42.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 146643968 bytes 2026-02-19T20:20:42.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 197044224 bytes 2026-02-19T20:20:42.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 197648384 bytes 2026-02-19T20:20:42.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 245337088 bytes 2026-02-19T20:20:43.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 263629824 bytes 2026-02-19T20:20:43.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 288373760 bytes 2026-02-19T20:20:43.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 310923264 bytes 2026-02-19T20:20:43.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 365340672 bytes 2026-02-19T20:20:43.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 429586432 bytes 2026-02-19T20:20:43.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 447612928 bytes 2026-02-19T20:20:43.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 456382464 bytes 2026-02-19T20:20:43.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 457781248 bytes 2026-02-19T20:20:43.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 512070656 bytes 2026-02-19T20:20:43.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 549001216 bytes 2026-02-19T20:20:43.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 579053568 bytes 2026-02-19T20:20:43.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 725543936 bytes 2026-02-19T20:20:43.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 799020032 bytes 2026-02-19T20:20:43.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 838026240 bytes 2026-02-19T20:20:43.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 865052672 bytes 2026-02-19T20:20:43.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 905472000 bytes 2026-02-19T20:20:43.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 906093568 bytes 2026-02-19T20:20:43.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 957985792 bytes 2026-02-19T20:20:44.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 983186432 bytes 2026-02-19T20:20:44.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1008373760 bytes 2026-02-19T20:20:44.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1030019072 bytes 2026-02-19T20:20:44.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1100214272 bytes 2026-02-19T20:20:44.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1109390336 bytes 2026-02-19T20:20:44.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1184648192 bytes 2026-02-19T20:20:44.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1285517312 bytes 2026-02-19T20:20:44.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1293392896 bytes 2026-02-19T20:20:44.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1298776064 bytes 2026-02-19T20:20:44.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1334696960 bytes 2026-02-19T20:20:44.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1351877632 bytes 2026-02-19T20:20:44.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1411467264 bytes 2026-02-19T20:20:44.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1437470720 bytes 2026-02-19T20:20:44.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1481137152 bytes 2026-02-19T20:20:44.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1503764480 bytes 2026-02-19T20:20:44.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1509485568 bytes 2026-02-19T20:20:44.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1520836608 bytes 2026-02-19T20:20:44.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1594487808 bytes 2026-02-19T20:20:45.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1635014656 bytes 2026-02-19T20:20:45.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1693598720 bytes 2026-02-19T20:20:45.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1716122624 bytes 2026-02-19T20:20:45.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1730100224 bytes 2026-02-19T20:20:45.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1795889152 bytes 2026-02-19T20:20:45.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1834536960 bytes 2026-02-19T20:20:45.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1891802112 bytes 2026-02-19T20:20:45.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1917661184 bytes 2026-02-19T20:20:45.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 1977485312 bytes 2026-02-19T20:20:45.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2000522240 bytes 2026-02-19T20:20:45.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2069634048 bytes 2026-02-19T20:20:45.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2087883776 bytes 2026-02-19T20:20:45.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2138597376 bytes 2026-02-19T20:20:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2210187264 bytes 2026-02-19T20:20:45.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2226163712 bytes 2026-02-19T20:20:45.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2298821632 bytes 2026-02-19T20:20:45.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2349209600 bytes 2026-02-19T20:20:46.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2436878336 bytes 2026-02-19T20:20:46.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2488085504 bytes 2026-02-19T20:20:46.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2548177920 bytes 2026-02-19T20:20:46.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2617063424 bytes 2026-02-19T20:20:46.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2709409792 bytes 2026-02-19T20:20:46.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2751403008 bytes 2026-02-19T20:20:46.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2806626304 bytes 2026-02-19T20:20:46.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2829111296 bytes 2026-02-19T20:20:46.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2852838400 bytes 2026-02-19T20:20:46.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2891726848 bytes 2026-02-19T20:20:46.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2946048000 bytes 2026-02-19T20:20:46.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3002311680 bytes 2026-02-19T20:20:46.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3073209344 bytes 2026-02-19T20:20:46.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3138749440 bytes 2026-02-19T20:20:47.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3196824576 bytes 2026-02-19T20:20:47.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3266860032 bytes 2026-02-19T20:20:47.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3337146368 bytes 2026-02-19T20:20:47.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3409595392 bytes 2026-02-19T20:20:47.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3466328064 bytes 2026-02-19T20:20:47.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3482183680 bytes 2026-02-19T20:20:47.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3482795008 bytes 2026-02-19T20:20:47.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3533048832 bytes 2026-02-19T20:20:47.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3613960192 bytes 2026-02-19T20:20:47.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3624048640 bytes 2026-02-19T20:20:47.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3642686464 bytes 2026-02-19T20:20:47.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3672735744 bytes 2026-02-19T20:20:47.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3695376384 bytes 2026-02-19T20:20:47.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3752924160 bytes 2026-02-19T20:20:47.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3760601088 bytes 2026-02-19T20:20:47.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3801619456 bytes 2026-02-19T20:20:47.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3889113088 bytes 2026-02-19T20:20:48.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3932492800 bytes 2026-02-19T20:20:48.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3933065216 bytes 2026-02-19T20:20:48.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3987022848 bytes 2026-02-19T20:20:48.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4015183872 bytes 2026-02-19T20:20:48.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4041956352 bytes 2026-02-19T20:20:48.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4087834624 bytes 2026-02-19T20:20:48.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4134034432 bytes 2026-02-19T20:20:48.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4168129536 bytes 2026-02-19T20:20:48.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4196887552 bytes 2026-02-19T20:20:48.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4268027904 bytes 2026-02-19T20:20:48.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4349316096 bytes 2026-02-19T20:20:48.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4351057920 bytes 2026-02-19T20:20:48.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4378484736 bytes 2026-02-19T20:20:48.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4439918592 bytes 2026-02-19T20:20:48.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4458101760 bytes 2026-02-19T20:20:48.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4471739392 bytes 2026-02-19T20:20:48.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4472370176 bytes 2026-02-19T20:20:48.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4547952640 bytes 2026-02-19T20:20:49.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4616982528 bytes 2026-02-19T20:20:49.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4643045376 bytes 2026-02-19T20:20:49.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4711143424 bytes 2026-02-19T20:20:49.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4714690560 bytes 2026-02-19T20:20:49.196 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4715316224 bytes 2026-02-19T20:20:49.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4723857408 bytes 2026-02-19T20:20:49.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4783635456 bytes 2026-02-19T20:20:49.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4784265216 bytes 2026-02-19T20:20:49.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4805293056 bytes 2026-02-19T20:20:49.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4837919744 bytes 2026-02-19T20:20:49.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4858648576 bytes 2026-02-19T20:20:49.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4955052032 bytes 2026-02-19T20:20:49.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4990619648 bytes 2026-02-19T20:20:49.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5082371072 bytes 2026-02-19T20:20:49.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5102900224 bytes 2026-02-19T20:20:49.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5151114240 bytes 2026-02-19T20:20:49.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5201804288 bytes 2026-02-19T20:20:49.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5221678080 bytes 2026-02-19T20:20:49.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5235601408 bytes 2026-02-19T20:20:50.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5312431104 bytes 2026-02-19T20:20:50.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5361619968 bytes 2026-02-19T20:20:50.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5373169664 bytes 2026-02-19T20:20:50.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5412728832 bytes 2026-02-19T20:20:50.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5426866176 bytes 2026-02-19T20:20:50.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5463556096 bytes 2026-02-19T20:20:50.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5514569728 bytes 2026-02-19T20:20:50.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5534417920 bytes 2026-02-19T20:20:50.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5584050176 bytes 2026-02-19T20:20:50.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5605001216 bytes 2026-02-19T20:20:50.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5605639168 bytes 2026-02-19T20:20:50.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5660000256 bytes 2026-02-19T20:20:50.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:20:50.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:20:50.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_15 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_15 snapshot 128M 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:20:50.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_15 128M 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:20:50.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:20:50.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:20:50.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:20:50.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_15 2026-02-19T20:20:50.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:20:50.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:50.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:20:50.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:20:50.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_15 snapshot 2026-02-19T20:20:50.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:50.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:50.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:20:50.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:20:50.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_15 snapshot 2026-02-19T20:20:51.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:20:51.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_15 2026-02-19T20:20:51.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_15': 2026-02-19T20:20:51.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:20:51.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:20:51.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:20:51.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1319b6352808 2026-02-19T20:20:51.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1319b6352808 2026-02-19T20:20:51.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:20:51.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:20:51.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:20:51.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:20:51.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:50 2026 2026-02-19T20:20:51.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:50 2026 2026-02-19T20:20:51.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:50 2026 2026-02-19T20:20:51.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:20:51.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:20:51.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: c0ee1cc7-c443-4e99-9bcc-31e4eda9076d 2026-02-19T20:20:51.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:20:51.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:20:51.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_15 2026-02-19T20:20:51.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:20:51.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:20:51.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:20:51.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:20:51.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:20:51.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:20:51.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_15 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:20:51.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 91448320 bytes 2026-02-19T20:20:51.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 185756672 bytes 2026-02-19T20:20:51.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 245621760 bytes 2026-02-19T20:20:51.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 268419072 bytes 2026-02-19T20:20:52.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 338385920 bytes 2026-02-19T20:20:52.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 402752512 bytes 2026-02-19T20:20:52.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 464603136 bytes 2026-02-19T20:20:52.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 552461312 bytes 2026-02-19T20:20:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 654326784 bytes 2026-02-19T20:20:52.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 689059840 bytes 2026-02-19T20:20:52.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 706692096 bytes 2026-02-19T20:20:52.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 771884032 bytes 2026-02-19T20:20:52.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 853032960 bytes 2026-02-19T20:20:52.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 906138624 bytes 2026-02-19T20:20:52.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 921150464 bytes 2026-02-19T20:20:52.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 960779264 bytes 2026-02-19T20:20:52.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 965432320 bytes 2026-02-19T20:20:52.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 1028815872 bytes 2026-02-19T20:20:52.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 1115786240 bytes 2026-02-19T20:20:53.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 1175880704 bytes 2026-02-19T20:20:53.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1210247168 bytes 2026-02-19T20:20:53.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1234619392 bytes 2026-02-19T20:20:53.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1296103424 bytes 2026-02-19T20:20:53.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1372623872 bytes 2026-02-19T20:20:53.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1373260800 bytes 2026-02-19T20:20:53.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1470303232 bytes 2026-02-19T20:20:53.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1536044032 bytes 2026-02-19T20:20:53.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1555129344 bytes 2026-02-19T20:20:53.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1608152064 bytes 2026-02-19T20:20:53.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1666304000 bytes 2026-02-19T20:20:53.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1685054464 bytes 2026-02-19T20:20:53.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1745582080 bytes 2026-02-19T20:20:53.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1802698752 bytes 2026-02-19T20:20:53.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1831084032 bytes 2026-02-19T20:20:53.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1842408448 bytes 2026-02-19T20:20:54.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1933118464 bytes 2026-02-19T20:20:54.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 2073079808 bytes 2026-02-19T20:20:54.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 2083464192 bytes 2026-02-19T20:20:54.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 2096776192 bytes 2026-02-19T20:20:54.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 2115147776 bytes 2026-02-19T20:20:54.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 2170540032 bytes 2026-02-19T20:20:54.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 2214331392 bytes 2026-02-19T20:20:54.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2230057984 bytes 2026-02-19T20:20:54.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2293157888 bytes 2026-02-19T20:20:54.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2323236864 bytes 2026-02-19T20:20:54.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2357121024 bytes 2026-02-19T20:20:54.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2432089088 bytes 2026-02-19T20:20:54.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2463929344 bytes 2026-02-19T20:20:54.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2514500608 bytes 2026-02-19T20:20:54.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2570291200 bytes 2026-02-19T20:20:54.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2592307200 bytes 2026-02-19T20:20:55.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2622964736 bytes 2026-02-19T20:20:55.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2642158592 bytes 2026-02-19T20:20:55.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2673107968 bytes 2026-02-19T20:20:55.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2717026304 bytes 2026-02-19T20:20:55.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2750049280 bytes 2026-02-19T20:20:55.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2775564288 bytes 2026-02-19T20:20:55.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2791068672 bytes 2026-02-19T20:20:55.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2832007168 bytes 2026-02-19T20:20:55.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2833659904 bytes 2026-02-19T20:20:55.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2891380736 bytes 2026-02-19T20:20:55.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2994330624 bytes 2026-02-19T20:20:55.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 3019522048 bytes 2026-02-19T20:20:55.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 3084438528 bytes 2026-02-19T20:20:55.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3107914752 bytes 2026-02-19T20:20:55.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3137929216 bytes 2026-02-19T20:20:55.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3189235712 bytes 2026-02-19T20:20:55.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3201687552 bytes 2026-02-19T20:20:55.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3203524608 bytes 2026-02-19T20:20:56.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3321090048 bytes 2026-02-19T20:20:56.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3387834368 bytes 2026-02-19T20:20:56.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3412800512 bytes 2026-02-19T20:20:56.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3527766016 bytes 2026-02-19T20:20:56.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3569613824 bytes 2026-02-19T20:20:56.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3573530624 bytes 2026-02-19T20:20:56.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3574136832 bytes 2026-02-19T20:20:56.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3615082496 bytes 2026-02-19T20:20:56.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3649591296 bytes 2026-02-19T20:20:56.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3757304832 bytes 2026-02-19T20:20:56.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3778063360 bytes 2026-02-19T20:20:56.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3847776256 bytes 2026-02-19T20:20:56.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3895894016 bytes 2026-02-19T20:20:56.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3950706688 bytes 2026-02-19T20:20:56.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3983016960 bytes 2026-02-19T20:20:57.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 4040956928 bytes 2026-02-19T20:20:57.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 4051612672 bytes 2026-02-19T20:20:57.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4087920640 bytes 2026-02-19T20:20:57.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4139015168 bytes 2026-02-19T20:20:57.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4204598272 bytes 2026-02-19T20:20:57.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4218590208 bytes 2026-02-19T20:20:57.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4314443776 bytes 2026-02-19T20:20:57.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4359497728 bytes 2026-02-19T20:20:57.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4360110080 bytes 2026-02-19T20:20:57.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4395865088 bytes 2026-02-19T20:20:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4433414144 bytes 2026-02-19T20:20:57.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4454138880 bytes 2026-02-19T20:20:57.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4500213760 bytes 2026-02-19T20:20:57.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4514785280 bytes 2026-02-19T20:20:57.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4521977856 bytes 2026-02-19T20:20:57.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4603328512 bytes 2026-02-19T20:20:57.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4636684288 bytes 2026-02-19T20:20:58.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4722813952 bytes 2026-02-19T20:20:58.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4747789312 bytes 2026-02-19T20:20:58.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4785229824 bytes 2026-02-19T20:20:58.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4801549312 bytes 2026-02-19T20:20:58.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4827175936 bytes 2026-02-19T20:20:58.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4927868928 bytes 2026-02-19T20:20:58.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4990158848 bytes 2026-02-19T20:20:58.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 5038532608 bytes 2026-02-19T20:20:58.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5039175680 bytes 2026-02-19T20:20:58.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5070775296 bytes 2026-02-19T20:20:58.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5089959936 bytes 2026-02-19T20:20:58.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5133340672 bytes 2026-02-19T20:20:58.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5133968384 bytes 2026-02-19T20:20:58.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5169829888 bytes 2026-02-19T20:20:58.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5203720192 bytes 2026-02-19T20:20:58.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5238828032 bytes 2026-02-19T20:20:58.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5298202624 bytes 2026-02-19T20:20:58.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5325349888 bytes 2026-02-19T20:20:59.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5374703616 bytes 2026-02-19T20:20:59.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5416925184 bytes 2026-02-19T20:20:59.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5475338240 bytes 2026-02-19T20:20:59.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5500635136 bytes 2026-02-19T20:20:59.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5530911744 bytes 2026-02-19T20:20:59.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5564153856 bytes 2026-02-19T20:20:59.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5659767808 bytes 2026-02-19T20:20:59.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5769689088 bytes 2026-02-19T20:20:59.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5806305280 bytes 2026-02-19T20:20:59.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5819269120 bytes 2026-02-19T20:20:59.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5861050368 bytes 2026-02-19T20:20:59.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5904956416 bytes 2026-02-19T20:20:59.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5939084288 bytes 2026-02-19T20:20:59.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5982862336 bytes 2026-02-19T20:20:59.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6002033664 bytes 2026-02-19T20:20:59.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6024623104 bytes 2026-02-19T20:20:59.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6047824896 bytes 2026-02-19T20:21:00.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6185723904 bytes 2026-02-19T20:21:00.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6206598144 bytes 2026-02-19T20:21:00.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6284711936 bytes 2026-02-19T20:21:00.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6294250496 bytes 2026-02-19T20:21:00.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6315990016 bytes 2026-02-19T20:21:00.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6344882176 bytes 2026-02-19T20:21:00.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6405830656 bytes 2026-02-19T20:21:00.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6418154496 bytes 2026-02-19T20:21:00.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6466385920 bytes 2026-02-19T20:21:00.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6534933504 bytes 2026-02-19T20:21:00.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6577394688 bytes 2026-02-19T20:21:00.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6699653120 bytes 2026-02-19T20:21:00.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6772022272 bytes 2026-02-19T20:21:00.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6794278912 bytes 2026-02-19T20:21:00.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6817501184 bytes 2026-02-19T20:21:01.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6867492864 bytes 2026-02-19T20:21:01.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6955265024 bytes 2026-02-19T20:21:01.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 7013403648 bytes 2026-02-19T20:21:01.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 7071272960 bytes 2026-02-19T20:21:01.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7132888064 bytes 2026-02-19T20:21:01.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7167893504 bytes 2026-02-19T20:21:01.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7325460480 bytes 2026-02-19T20:21:01.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7345646592 bytes 2026-02-19T20:21:01.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7356170240 bytes 2026-02-19T20:21:01.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7407080448 bytes 2026-02-19T20:21:01.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7407689728 bytes 2026-02-19T20:21:01.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7427874816 bytes 2026-02-19T20:21:01.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7482008576 bytes 2026-02-19T20:21:01.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7543533568 bytes 2026-02-19T20:21:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7591565312 bytes 2026-02-19T20:21:01.957 INFO:tasks.ceph.cluster2.osd.1.trial080.stderr:2026-02-19T20:21:01.954+0000 7f1a949a9640 -1 reset not still connected to 0x4681a00 2026-02-19T20:21:02.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7655297024 bytes 2026-02-19T20:21:02.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7692466176 bytes 2026-02-19T20:21:02.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7719236608 bytes 2026-02-19T20:21:02.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7746495488 bytes 2026-02-19T20:21:02.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7794636800 bytes 2026-02-19T20:21:02.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7866280960 bytes 2026-02-19T20:21:02.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7923396608 bytes 2026-02-19T20:21:02.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7994807296 bytes 2026-02-19T20:21:02.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 8001173504 bytes 2026-02-19T20:21:02.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 8029095936 bytes 2026-02-19T20:21:02.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 8086456320 bytes 2026-02-19T20:21:02.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 8142604288 bytes 2026-02-19T20:21:02.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 8187668480 bytes 2026-02-19T20:21:02.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 8230888448 bytes 2026-02-19T20:21:02.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 8289828864 bytes 2026-02-19T20:21:02.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 8351079424 bytes 2026-02-19T20:21:03.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 8390924288 bytes 2026-02-19T20:21:03.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 8434488320 bytes 2026-02-19T20:21:03.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 8470147072 bytes 2026-02-19T20:21:03.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8523409408 bytes 2026-02-19T20:21:03.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8545310720 bytes 2026-02-19T20:21:03.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8581000192 bytes 2026-02-19T20:21:03.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8605125632 bytes 2026-02-19T20:21:03.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8662847488 bytes 2026-02-19T20:21:03.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8694142976 bytes 2026-02-19T20:21:03.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8740433920 bytes 2026-02-19T20:21:03.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8753987584 bytes 2026-02-19T20:21:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:21:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:21:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:21:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:21:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_16 2026-02-19T20:21:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_16 snapshot 128M 2026-02-19T20:21:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:21:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_16 128M 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:21:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:03.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_16 2026-02-19T20:21:03.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:21:03.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:21:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_16 snapshot 2026-02-19T20:21:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:21:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_16 snapshot 2026-02-19T20:21:04.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:21:04.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_16 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_16': 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 132594a274e5 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.132594a274e5 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:21:04.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:21:04.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:03 2026 2026-02-19T20:21:04.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:03 2026 2026-02-19T20:21:04.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:03 2026 2026-02-19T20:21:04.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:21:04.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:21:04.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 97f2f53e-69ec-439c-87a1-d3ef7d099edc 2026-02-19T20:21:04.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:21:04.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:21:04.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_16 2026-02-19T20:21:04.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:04.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:04.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:21:04.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:21:04.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:21:04.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:21:04.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_16 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:21:04.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 64643072 bytes 2026-02-19T20:21:04.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 139965440 bytes 2026-02-19T20:21:04.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 159907840 bytes 2026-02-19T20:21:04.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 179460096 bytes 2026-02-19T20:21:04.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 184580096 bytes 2026-02-19T20:21:04.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 231392256 bytes 2026-02-19T20:21:05.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 258503680 bytes 2026-02-19T20:21:05.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 337836032 bytes 2026-02-19T20:21:05.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 360547328 bytes 2026-02-19T20:21:05.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 385210368 bytes 2026-02-19T20:21:05.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 448287744 bytes 2026-02-19T20:21:05.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 467081216 bytes 2026-02-19T20:21:05.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 504211456 bytes 2026-02-19T20:21:05.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 527545344 bytes 2026-02-19T20:21:05.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 540882944 bytes 2026-02-19T20:21:05.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 567979008 bytes 2026-02-19T20:21:05.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 620607488 bytes 2026-02-19T20:21:05.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 674698240 bytes 2026-02-19T20:21:05.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 724671488 bytes 2026-02-19T20:21:05.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 738248704 bytes 2026-02-19T20:21:05.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 814670848 bytes 2026-02-19T20:21:05.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 843493376 bytes 2026-02-19T20:21:06.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 949317632 bytes 2026-02-19T20:21:06.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 982159360 bytes 2026-02-19T20:21:06.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1020271616 bytes 2026-02-19T20:21:06.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1056006144 bytes 2026-02-19T20:21:06.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1092313088 bytes 2026-02-19T20:21:06.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1139408896 bytes 2026-02-19T20:21:06.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1207147520 bytes 2026-02-19T20:21:06.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1274521600 bytes 2026-02-19T20:21:06.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1288686592 bytes 2026-02-19T20:21:06.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1318761472 bytes 2026-02-19T20:21:06.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1339418624 bytes 2026-02-19T20:21:06.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1356607488 bytes 2026-02-19T20:21:06.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1441113088 bytes 2026-02-19T20:21:06.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1521908736 bytes 2026-02-19T20:21:06.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1579777024 bytes 2026-02-19T20:21:07.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1616083968 bytes 2026-02-19T20:21:07.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1628658688 bytes 2026-02-19T20:21:07.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1677058048 bytes 2026-02-19T20:21:07.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1713003520 bytes 2026-02-19T20:21:07.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1761231872 bytes 2026-02-19T20:21:07.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1804524544 bytes 2026-02-19T20:21:07.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1854555136 bytes 2026-02-19T20:21:07.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1933396992 bytes 2026-02-19T20:21:07.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1964265472 bytes 2026-02-19T20:21:07.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2035023872 bytes 2026-02-19T20:21:07.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2073972736 bytes 2026-02-19T20:21:07.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2145339392 bytes 2026-02-19T20:21:07.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2197160960 bytes 2026-02-19T20:21:07.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2197780480 bytes 2026-02-19T20:21:07.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2235154432 bytes 2026-02-19T20:21:08.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2298632192 bytes 2026-02-19T20:21:08.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2332093440 bytes 2026-02-19T20:21:08.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2372557824 bytes 2026-02-19T20:21:08.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2498207744 bytes 2026-02-19T20:21:08.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2541284352 bytes 2026-02-19T20:21:08.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2566021120 bytes 2026-02-19T20:21:08.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2601500672 bytes 2026-02-19T20:21:08.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2686000128 bytes 2026-02-19T20:21:08.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2733336576 bytes 2026-02-19T20:21:08.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2759207936 bytes 2026-02-19T20:21:08.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2818988032 bytes 2026-02-19T20:21:08.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2856188928 bytes 2026-02-19T20:21:08.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2923276288 bytes 2026-02-19T20:21:08.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2946674688 bytes 2026-02-19T20:21:08.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2964580352 bytes 2026-02-19T20:21:08.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2965164032 bytes 2026-02-19T20:21:09.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3054081024 bytes 2026-02-19T20:21:09.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3073863680 bytes 2026-02-19T20:21:09.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3086244864 bytes 2026-02-19T20:21:09.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3095341056 bytes 2026-02-19T20:21:09.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3146562560 bytes 2026-02-19T20:21:09.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3168658432 bytes 2026-02-19T20:21:09.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3227352064 bytes 2026-02-19T20:21:09.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3246369792 bytes 2026-02-19T20:21:09.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3338434560 bytes 2026-02-19T20:21:09.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3407240192 bytes 2026-02-19T20:21:09.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3433095168 bytes 2026-02-19T20:21:09.762 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3472582656 bytes 2026-02-19T20:21:09.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3528851456 bytes 2026-02-19T20:21:09.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3592782848 bytes 2026-02-19T20:21:09.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3603279872 bytes 2026-02-19T20:21:10.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3650283520 bytes 2026-02-19T20:21:10.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3729770496 bytes 2026-02-19T20:21:10.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3780796416 bytes 2026-02-19T20:21:10.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3818387456 bytes 2026-02-19T20:21:10.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3820272640 bytes 2026-02-19T20:21:10.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3848053760 bytes 2026-02-19T20:21:10.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3916030976 bytes 2026-02-19T20:21:10.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3955229696 bytes 2026-02-19T20:21:10.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4015805440 bytes 2026-02-19T20:21:10.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4035851264 bytes 2026-02-19T20:21:10.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4053303296 bytes 2026-02-19T20:21:10.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4099434496 bytes 2026-02-19T20:21:10.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4146101248 bytes 2026-02-19T20:21:10.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4186080256 bytes 2026-02-19T20:21:10.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4214614016 bytes 2026-02-19T20:21:10.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4328061952 bytes 2026-02-19T20:21:11.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4371259392 bytes 2026-02-19T20:21:11.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4390284288 bytes 2026-02-19T20:21:11.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4469322752 bytes 2026-02-19T20:21:11.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4534910976 bytes 2026-02-19T20:21:11.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4542320640 bytes 2026-02-19T20:21:11.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4590953472 bytes 2026-02-19T20:21:11.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4594064384 bytes 2026-02-19T20:21:11.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4599748608 bytes 2026-02-19T20:21:11.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4668156928 bytes 2026-02-19T20:21:11.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4718074880 bytes 2026-02-19T20:21:11.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4734971904 bytes 2026-02-19T20:21:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4759539712 bytes 2026-02-19T20:21:11.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4785599488 bytes 2026-02-19T20:21:11.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4821831680 bytes 2026-02-19T20:21:11.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4857073664 bytes 2026-02-19T20:21:11.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4911139840 bytes 2026-02-19T20:21:11.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4948463616 bytes 2026-02-19T20:21:12.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4973209600 bytes 2026-02-19T20:21:12.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4987879424 bytes 2026-02-19T20:21:12.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5052356608 bytes 2026-02-19T20:21:12.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5095340032 bytes 2026-02-19T20:21:12.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5139219456 bytes 2026-02-19T20:21:12.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5221452800 bytes 2026-02-19T20:21:12.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5222076416 bytes 2026-02-19T20:21:12.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5250843648 bytes 2026-02-19T20:21:12.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5278879744 bytes 2026-02-19T20:21:12.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5318907904 bytes 2026-02-19T20:21:12.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5319529472 bytes 2026-02-19T20:21:12.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5338435584 bytes 2026-02-19T20:21:12.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5407004672 bytes 2026-02-19T20:21:12.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5476099072 bytes 2026-02-19T20:21:12.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5512522752 bytes 2026-02-19T20:21:12.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5522044928 bytes 2026-02-19T20:21:12.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5586554880 bytes 2026-02-19T20:21:13.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5633991680 bytes 2026-02-19T20:21:13.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5727112192 bytes 2026-02-19T20:21:13.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5729885184 bytes 2026-02-19T20:21:13.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5806860288 bytes 2026-02-19T20:21:13.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5881316352 bytes 2026-02-19T20:21:13.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5944669184 bytes 2026-02-19T20:21:13.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5953479680 bytes 2026-02-19T20:21:13.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5994684416 bytes 2026-02-19T20:21:13.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6037791744 bytes 2026-02-19T20:21:13.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6063395840 bytes 2026-02-19T20:21:13.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6108611584 bytes 2026-02-19T20:21:13.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6124012544 bytes 2026-02-19T20:21:13.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6235278336 bytes 2026-02-19T20:21:13.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6306991104 bytes 2026-02-19T20:21:13.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6331358208 bytes 2026-02-19T20:21:14.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6372943872 bytes 2026-02-19T20:21:14.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6443160576 bytes 2026-02-19T20:21:14.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6491014144 bytes 2026-02-19T20:21:14.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6509444096 bytes 2026-02-19T20:21:14.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6524505088 bytes 2026-02-19T20:21:14.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6542870528 bytes 2026-02-19T20:21:14.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6568946688 bytes 2026-02-19T20:21:14.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6610906112 bytes 2026-02-19T20:21:14.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6683246592 bytes 2026-02-19T20:21:14.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6728457216 bytes 2026-02-19T20:21:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6767408128 bytes 2026-02-19T20:21:14.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6840231936 bytes 2026-02-19T20:21:14.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6899934208 bytes 2026-02-19T20:21:14.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6930422784 bytes 2026-02-19T20:21:14.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6992691200 bytes 2026-02-19T20:21:15.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6997495808 bytes 2026-02-19T20:21:15.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7029940224 bytes 2026-02-19T20:21:15.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7087455232 bytes 2026-02-19T20:21:15.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7106651136 bytes 2026-02-19T20:21:15.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7222427648 bytes 2026-02-19T20:21:15.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7278113792 bytes 2026-02-19T20:21:15.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7285023744 bytes 2026-02-19T20:21:15.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7291649024 bytes 2026-02-19T20:21:15.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7316954112 bytes 2026-02-19T20:21:15.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7369234432 bytes 2026-02-19T20:21:15.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7435515904 bytes 2026-02-19T20:21:15.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7442392064 bytes 2026-02-19T20:21:15.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7478187008 bytes 2026-02-19T20:21:15.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7479207936 bytes 2026-02-19T20:21:15.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7516508160 bytes 2026-02-19T20:21:15.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7545066496 bytes 2026-02-19T20:21:16.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7678475264 bytes 2026-02-19T20:21:16.049 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7689502720 bytes 2026-02-19T20:21:16.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7695430656 bytes 2026-02-19T20:21:16.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7741809664 bytes 2026-02-19T20:21:16.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7821172736 bytes 2026-02-19T20:21:16.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7844371456 bytes 2026-02-19T20:21:16.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7878938624 bytes 2026-02-19T20:21:16.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7991611392 bytes 2026-02-19T20:21:16.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8017249280 bytes 2026-02-19T20:21:16.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8017862656 bytes 2026-02-19T20:21:16.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8105461760 bytes 2026-02-19T20:21:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8124146688 bytes 2026-02-19T20:21:16.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8211674112 bytes 2026-02-19T20:21:16.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8233183232 bytes 2026-02-19T20:21:16.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8261280768 bytes 2026-02-19T20:21:16.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8315166720 bytes 2026-02-19T20:21:17.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8403134464 bytes 2026-02-19T20:21:17.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8457822208 bytes 2026-02-19T20:21:17.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8482771968 bytes 2026-02-19T20:21:17.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8534468608 bytes 2026-02-19T20:21:17.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8569932800 bytes 2026-02-19T20:21:17.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8584784896 bytes 2026-02-19T20:21:17.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8650684416 bytes 2026-02-19T20:21:17.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8707137536 bytes 2026-02-19T20:21:17.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8733733888 bytes 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_17 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_17 snapshot 128M 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:17.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_17 128M 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:17.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_17 2026-02-19T20:21:17.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:21:17.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:17.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:21:17.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:17.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_17 snapshot 2026-02-19T20:21:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:21:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_17 snapshot 2026-02-19T20:21:18.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:21:18.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_17 2026-02-19T20:21:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_17': 2026-02-19T20:21:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:21:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:21:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:21:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 13313886d79 2026-02-19T20:21:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.13313886d79 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:17 2026 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:17 2026 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:17 2026 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 412c65be-fc7c-4de7-ab85-528c3d9910a2 2026-02-19T20:21:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:21:18.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:21:18.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_17 2026-02-19T20:21:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:21:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:21:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=14 2026-02-19T20:21:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:21:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_17 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:21:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 36052992 bytes 2026-02-19T20:21:18.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 64450560 bytes 2026-02-19T20:21:18.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 115361792 bytes 2026-02-19T20:21:18.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 128761856 bytes 2026-02-19T20:21:18.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 169549824 bytes 2026-02-19T20:21:18.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 195124224 bytes 2026-02-19T20:21:18.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 227689472 bytes 2026-02-19T20:21:19.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 249294848 bytes 2026-02-19T20:21:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 279100416 bytes 2026-02-19T20:21:19.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 289888256 bytes 2026-02-19T20:21:19.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 346343424 bytes 2026-02-19T20:21:19.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 407665664 bytes 2026-02-19T20:21:19.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 447925248 bytes 2026-02-19T20:21:19.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 452446208 bytes 2026-02-19T20:21:19.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 471958528 bytes 2026-02-19T20:21:19.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 530772992 bytes 2026-02-19T20:21:19.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 565140480 bytes 2026-02-19T20:21:19.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 655792128 bytes 2026-02-19T20:21:19.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 683649024 bytes 2026-02-19T20:21:19.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 717368320 bytes 2026-02-19T20:21:19.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 768530432 bytes 2026-02-19T20:21:20.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 806038528 bytes 2026-02-19T20:21:20.103 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 807727104 bytes 2026-02-19T20:21:20.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 939661312 bytes 2026-02-19T20:21:20.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 968683520 bytes 2026-02-19T20:21:20.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1025541120 bytes 2026-02-19T20:21:20.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1046347776 bytes 2026-02-19T20:21:20.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1046973440 bytes 2026-02-19T20:21:20.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1057605632 bytes 2026-02-19T20:21:20.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1104707584 bytes 2026-02-19T20:21:20.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1156229120 bytes 2026-02-19T20:21:20.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1177769984 bytes 2026-02-19T20:21:20.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1241024512 bytes 2026-02-19T20:21:20.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1246358528 bytes 2026-02-19T20:21:20.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1269962752 bytes 2026-02-19T20:21:20.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1334649856 bytes 2026-02-19T20:21:20.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1390046208 bytes 2026-02-19T20:21:21.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1411098624 bytes 2026-02-19T20:21:21.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1460739072 bytes 2026-02-19T20:21:21.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1523611648 bytes 2026-02-19T20:21:21.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1572813824 bytes 2026-02-19T20:21:21.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1578424320 bytes 2026-02-19T20:21:21.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1579065344 bytes 2026-02-19T20:21:21.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1680242688 bytes 2026-02-19T20:21:21.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1700276224 bytes 2026-02-19T20:21:21.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1736276992 bytes 2026-02-19T20:21:21.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1823449088 bytes 2026-02-19T20:21:21.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1851406336 bytes 2026-02-19T20:21:21.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1884778496 bytes 2026-02-19T20:21:21.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1886491648 bytes 2026-02-19T20:21:21.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1920002048 bytes 2026-02-19T20:21:21.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 1937760256 bytes 2026-02-19T20:21:21.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 1972739072 bytes 2026-02-19T20:21:21.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2006531072 bytes 2026-02-19T20:21:21.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2062159872 bytes 2026-02-19T20:21:22.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2092401664 bytes 2026-02-19T20:21:22.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2122086400 bytes 2026-02-19T20:21:22.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2177181696 bytes 2026-02-19T20:21:22.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2177787904 bytes 2026-02-19T20:21:22.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2217203712 bytes 2026-02-19T20:21:22.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2217823232 bytes 2026-02-19T20:21:22.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2230363136 bytes 2026-02-19T20:21:22.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2238622720 bytes 2026-02-19T20:21:22.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2302364672 bytes 2026-02-19T20:21:22.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2325485568 bytes 2026-02-19T20:21:22.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2332817408 bytes 2026-02-19T20:21:22.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2379157504 bytes 2026-02-19T20:21:22.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2445047808 bytes 2026-02-19T20:21:22.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2485438464 bytes 2026-02-19T20:21:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2515714048 bytes 2026-02-19T20:21:22.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2585133056 bytes 2026-02-19T20:21:22.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2646807552 bytes 2026-02-19T20:21:22.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2650572800 bytes 2026-02-19T20:21:23.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2659704832 bytes 2026-02-19T20:21:23.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2702032896 bytes 2026-02-19T20:21:23.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2766785536 bytes 2026-02-19T20:21:23.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 2835377152 bytes 2026-02-19T20:21:23.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 2873019392 bytes 2026-02-19T20:21:23.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 2910104576 bytes 2026-02-19T20:21:23.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 2936400896 bytes 2026-02-19T20:21:23.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3004862464 bytes 2026-02-19T20:21:23.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3056844800 bytes 2026-02-19T20:21:23.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3127349248 bytes 2026-02-19T20:21:23.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3157844992 bytes 2026-02-19T20:21:23.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3183473664 bytes 2026-02-19T20:21:23.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3219618816 bytes 2026-02-19T20:21:23.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3298417664 bytes 2026-02-19T20:21:23.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3311143936 bytes 2026-02-19T20:21:24.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3351688192 bytes 2026-02-19T20:21:24.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3388767232 bytes 2026-02-19T20:21:24.103 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3391441920 bytes 2026-02-19T20:21:24.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3411922944 bytes 2026-02-19T20:21:24.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3445526528 bytes 2026-02-19T20:21:24.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3446138880 bytes 2026-02-19T20:21:24.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3448108032 bytes 2026-02-19T20:21:24.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3483445248 bytes 2026-02-19T20:21:24.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3511138304 bytes 2026-02-19T20:21:24.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3514116096 bytes 2026-02-19T20:21:24.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3589485568 bytes 2026-02-19T20:21:24.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 3648024576 bytes 2026-02-19T20:21:24.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 3713726464 bytes 2026-02-19T20:21:24.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 3756272640 bytes 2026-02-19T20:21:24.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 3821084672 bytes 2026-02-19T20:21:24.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 3864999936 bytes 2026-02-19T20:21:24.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 3933634560 bytes 2026-02-19T20:21:24.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 3934263296 bytes 2026-02-19T20:21:25.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 3974667264 bytes 2026-02-19T20:21:25.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 3996863488 bytes 2026-02-19T20:21:25.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4001531904 bytes 2026-02-19T20:21:25.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4061915136 bytes 2026-02-19T20:21:25.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4070301696 bytes 2026-02-19T20:21:25.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4102018048 bytes 2026-02-19T20:21:25.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4102660096 bytes 2026-02-19T20:21:25.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4145902592 bytes 2026-02-19T20:21:25.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4186483712 bytes 2026-02-19T20:21:25.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4210623488 bytes 2026-02-19T20:21:25.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4213970944 bytes 2026-02-19T20:21:25.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4238410752 bytes 2026-02-19T20:21:25.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4255551488 bytes 2026-02-19T20:21:25.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4292104192 bytes 2026-02-19T20:21:25.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4353609728 bytes 2026-02-19T20:21:25.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4391985152 bytes 2026-02-19T20:21:25.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4416484352 bytes 2026-02-19T20:21:25.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4441766912 bytes 2026-02-19T20:21:25.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 4459777024 bytes 2026-02-19T20:21:26.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 4496185344 bytes 2026-02-19T20:21:26.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 4525138944 bytes 2026-02-19T20:21:26.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 4562854912 bytes 2026-02-19T20:21:26.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 4623940608 bytes 2026-02-19T20:21:26.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 4651578368 bytes 2026-02-19T20:21:26.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 4729446400 bytes 2026-02-19T20:21:26.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 4777698304 bytes 2026-02-19T20:21:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 4818112512 bytes 2026-02-19T20:21:26.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 4854441984 bytes 2026-02-19T20:21:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 4897641472 bytes 2026-02-19T20:21:26.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 4925415424 bytes 2026-02-19T20:21:26.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5020276736 bytes 2026-02-19T20:21:26.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5073781760 bytes 2026-02-19T20:21:26.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5116650496 bytes 2026-02-19T20:21:27.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5140463616 bytes 2026-02-19T20:21:27.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5153422336 bytes 2026-02-19T20:21:27.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5198088192 bytes 2026-02-19T20:21:27.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5198795776 bytes 2026-02-19T20:21:27.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5221585920 bytes 2026-02-19T20:21:27.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5226464256 bytes 2026-02-19T20:21:27.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5242776576 bytes 2026-02-19T20:21:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5261755392 bytes 2026-02-19T20:21:27.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5302651904 bytes 2026-02-19T20:21:27.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 5334619136 bytes 2026-02-19T20:21:27.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 5375900672 bytes 2026-02-19T20:21:27.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 5416704000 bytes 2026-02-19T20:21:27.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 5484605440 bytes 2026-02-19T20:21:27.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 5534001152 bytes 2026-02-19T20:21:27.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 5548215296 bytes 2026-02-19T20:21:27.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 5571720192 bytes 2026-02-19T20:21:27.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 5634016256 bytes 2026-02-19T20:21:27.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 5712890880 bytes 2026-02-19T20:21:28.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 5777695744 bytes 2026-02-19T20:21:28.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 5816995840 bytes 2026-02-19T20:21:28.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 5870268416 bytes 2026-02-19T20:21:28.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 5896124416 bytes 2026-02-19T20:21:28.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 5932785664 bytes 2026-02-19T20:21:28.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 5963746304 bytes 2026-02-19T20:21:28.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 5987605504 bytes 2026-02-19T20:21:28.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6029798400 bytes 2026-02-19T20:21:28.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6085301248 bytes 2026-02-19T20:21:28.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6110848000 bytes 2026-02-19T20:21:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6180063232 bytes 2026-02-19T20:21:28.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6224861184 bytes 2026-02-19T20:21:28.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6334487552 bytes 2026-02-19T20:21:28.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6389380096 bytes 2026-02-19T20:21:28.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 6458639360 bytes 2026-02-19T20:21:29.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 6531720192 bytes 2026-02-19T20:21:29.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 6555045888 bytes 2026-02-19T20:21:29.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 6569729024 bytes 2026-02-19T20:21:29.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 6612847616 bytes 2026-02-19T20:21:29.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 6635600896 bytes 2026-02-19T20:21:29.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 6696796160 bytes 2026-02-19T20:21:29.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 6697422848 bytes 2026-02-19T20:21:29.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 6763022336 bytes 2026-02-19T20:21:29.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 6780299264 bytes 2026-02-19T20:21:29.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 6830435328 bytes 2026-02-19T20:21:29.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 6851367936 bytes 2026-02-19T20:21:29.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 6899416064 bytes 2026-02-19T20:21:29.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 6954870784 bytes 2026-02-19T20:21:29.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7002149888 bytes 2026-02-19T20:21:29.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7072868352 bytes 2026-02-19T20:21:29.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7101036544 bytes 2026-02-19T20:21:30.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7164030976 bytes 2026-02-19T20:21:30.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7183288320 bytes 2026-02-19T20:21:30.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7238899712 bytes 2026-02-19T20:21:30.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7255071744 bytes 2026-02-19T20:21:30.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7322479616 bytes 2026-02-19T20:21:30.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 7333786624 bytes 2026-02-19T20:21:30.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 7360154624 bytes 2026-02-19T20:21:30.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 7413522432 bytes 2026-02-19T20:21:30.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 7461025792 bytes 2026-02-19T20:21:30.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 7478587392 bytes 2026-02-19T20:21:30.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 7524842496 bytes 2026-02-19T20:21:30.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 7554598912 bytes 2026-02-19T20:21:30.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 7590436864 bytes 2026-02-19T20:21:31.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 7630128128 bytes 2026-02-19T20:21:31.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 7682486272 bytes 2026-02-19T20:21:31.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 7698449408 bytes 2026-02-19T20:21:31.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 7751330816 bytes 2026-02-19T20:21:31.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 7825247232 bytes 2026-02-19T20:21:31.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 7857051648 bytes 2026-02-19T20:21:31.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 7883719680 bytes 2026-02-19T20:21:31.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 7962405888 bytes 2026-02-19T20:21:31.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 8010076160 bytes 2026-02-19T20:21:31.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 8043464704 bytes 2026-02-19T20:21:31.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 8105470976 bytes 2026-02-19T20:21:31.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 8129598464 bytes 2026-02-19T20:21:31.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 8179927040 bytes 2026-02-19T20:21:31.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 8207055872 bytes 2026-02-19T20:21:31.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 8243720192 bytes 2026-02-19T20:21:31.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 8244356096 bytes 2026-02-19T20:21:32.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 8284545024 bytes 2026-02-19T20:21:32.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 8314064896 bytes 2026-02-19T20:21:32.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 8421996544 bytes 2026-02-19T20:21:32.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 8452155392 bytes 2026-02-19T20:21:32.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 8466381824 bytes 2026-02-19T20:21:32.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 8507990016 bytes 2026-02-19T20:21:32.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 8512189440 bytes 2026-02-19T20:21:32.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 8535225344 bytes 2026-02-19T20:21:32.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22600 IOs, 8589832192 bytes 2026-02-19T20:21:32.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22700 IOs, 8609284096 bytes 2026-02-19T20:21:32.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:21:32.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:21:32.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:21:32.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:32.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:21:32.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_18 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_18 snapshot 128M 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_18 128M 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:32.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_18 2026-02-19T20:21:32.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:21:32.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:32.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:21:32.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:32.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_18 snapshot 2026-02-19T20:21:32.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:32.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:32.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:21:32.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:32.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_18 snapshot 2026-02-19T20:21:33.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:21:33.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_18 2026-02-19T20:21:33.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_18': 2026-02-19T20:21:33.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 133deb1858a9 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.133deb1858a9 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:32 2026 2026-02-19T20:21:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:32 2026 2026-02-19T20:21:33.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:32 2026 2026-02-19T20:21:33.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:21:33.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:21:33.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: b08bc81f-704a-41ce-b79c-ab730842d310 2026-02-19T20:21:33.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:21:33.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:21:33.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_18 2026-02-19T20:21:33.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:33.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:33.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:21:33.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:21:33.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=7 2026-02-19T20:21:33.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:21:33.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_18 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:21:33.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 47798272 bytes 2026-02-19T20:21:33.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 118856704 bytes 2026-02-19T20:21:33.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 166379520 bytes 2026-02-19T20:21:33.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 227758080 bytes 2026-02-19T20:21:33.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 281077760 bytes 2026-02-19T20:21:34.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 342215680 bytes 2026-02-19T20:21:34.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 379136000 bytes 2026-02-19T20:21:34.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 454328320 bytes 2026-02-19T20:21:34.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 539241472 bytes 2026-02-19T20:21:34.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 582777856 bytes 2026-02-19T20:21:34.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 634585088 bytes 2026-02-19T20:21:34.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 749129728 bytes 2026-02-19T20:21:34.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 794842112 bytes 2026-02-19T20:21:34.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 882206720 bytes 2026-02-19T20:21:34.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 918548480 bytes 2026-02-19T20:21:34.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 964414464 bytes 2026-02-19T20:21:34.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 987539456 bytes 2026-02-19T20:21:34.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 1018527744 bytes 2026-02-19T20:21:34.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 1097957376 bytes 2026-02-19T20:21:35.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 1154369536 bytes 2026-02-19T20:21:35.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1197378560 bytes 2026-02-19T20:21:35.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1267357696 bytes 2026-02-19T20:21:35.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1339640832 bytes 2026-02-19T20:21:35.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1391141888 bytes 2026-02-19T20:21:35.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1440395264 bytes 2026-02-19T20:21:35.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1471370240 bytes 2026-02-19T20:21:35.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1495049216 bytes 2026-02-19T20:21:35.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1559308288 bytes 2026-02-19T20:21:35.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1621364736 bytes 2026-02-19T20:21:35.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1621982208 bytes 2026-02-19T20:21:35.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1675412480 bytes 2026-02-19T20:21:35.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1760703488 bytes 2026-02-19T20:21:35.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1773724672 bytes 2026-02-19T20:21:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1803851776 bytes 2026-02-19T20:21:35.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1891622912 bytes 2026-02-19T20:21:36.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1950648320 bytes 2026-02-19T20:21:36.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1997298688 bytes 2026-02-19T20:21:36.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 2022786048 bytes 2026-02-19T20:21:36.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 2060233728 bytes 2026-02-19T20:21:36.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 2160337920 bytes 2026-02-19T20:21:36.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 2196561920 bytes 2026-02-19T20:21:36.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 2235476992 bytes 2026-02-19T20:21:36.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2244091904 bytes 2026-02-19T20:21:36.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2285568000 bytes 2026-02-19T20:21:36.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2372180992 bytes 2026-02-19T20:21:36.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2402914304 bytes 2026-02-19T20:21:36.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2475592704 bytes 2026-02-19T20:21:36.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2514081792 bytes 2026-02-19T20:21:36.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2550560768 bytes 2026-02-19T20:21:36.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2592668672 bytes 2026-02-19T20:21:36.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2703293440 bytes 2026-02-19T20:21:37.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2738029568 bytes 2026-02-19T20:21:37.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2770946048 bytes 2026-02-19T20:21:37.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2791793664 bytes 2026-02-19T20:21:37.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2859273216 bytes 2026-02-19T20:21:37.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2907164672 bytes 2026-02-19T20:21:37.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2907787264 bytes 2026-02-19T20:21:37.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2916329472 bytes 2026-02-19T20:21:37.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2941290496 bytes 2026-02-19T20:21:37.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2955156480 bytes 2026-02-19T20:21:37.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2989172736 bytes 2026-02-19T20:21:37.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 3003585536 bytes 2026-02-19T20:21:37.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 3019414528 bytes 2026-02-19T20:21:37.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 3076803584 bytes 2026-02-19T20:21:37.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3109251072 bytes 2026-02-19T20:21:37.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3173776384 bytes 2026-02-19T20:21:37.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3238587392 bytes 2026-02-19T20:21:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3259520000 bytes 2026-02-19T20:21:37.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3273204736 bytes 2026-02-19T20:21:37.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3311193088 bytes 2026-02-19T20:21:38.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3328686080 bytes 2026-02-19T20:21:38.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3393341440 bytes 2026-02-19T20:21:38.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3413393408 bytes 2026-02-19T20:21:38.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3496109056 bytes 2026-02-19T20:21:38.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3496727552 bytes 2026-02-19T20:21:38.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3554917376 bytes 2026-02-19T20:21:38.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3646373888 bytes 2026-02-19T20:21:38.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3697373184 bytes 2026-02-19T20:21:38.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3723157504 bytes 2026-02-19T20:21:38.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3749578752 bytes 2026-02-19T20:21:38.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3808977920 bytes 2026-02-19T20:21:38.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3865623552 bytes 2026-02-19T20:21:38.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3889753088 bytes 2026-02-19T20:21:38.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3972096000 bytes 2026-02-19T20:21:38.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 4042195968 bytes 2026-02-19T20:21:38.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 4111046656 bytes 2026-02-19T20:21:39.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4151266304 bytes 2026-02-19T20:21:39.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4167966720 bytes 2026-02-19T20:21:39.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4213665792 bytes 2026-02-19T20:21:39.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4271496192 bytes 2026-02-19T20:21:39.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4315476992 bytes 2026-02-19T20:21:39.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4349336576 bytes 2026-02-19T20:21:39.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4379232256 bytes 2026-02-19T20:21:39.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4405868544 bytes 2026-02-19T20:21:39.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4433839104 bytes 2026-02-19T20:21:39.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4496480256 bytes 2026-02-19T20:21:39.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4583154688 bytes 2026-02-19T20:21:39.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4610385920 bytes 2026-02-19T20:21:39.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4671335424 bytes 2026-02-19T20:21:39.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4763968512 bytes 2026-02-19T20:21:39.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4823487488 bytes 2026-02-19T20:21:39.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4862868480 bytes 2026-02-19T20:21:40.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4915962880 bytes 2026-02-19T20:21:40.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4977882112 bytes 2026-02-19T20:21:40.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4988837888 bytes 2026-02-19T20:21:40.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 5008474112 bytes 2026-02-19T20:21:40.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 5028562944 bytes 2026-02-19T20:21:40.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 5064202240 bytes 2026-02-19T20:21:40.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 5088589824 bytes 2026-02-19T20:21:40.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5089221632 bytes 2026-02-19T20:21:40.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5198401536 bytes 2026-02-19T20:21:40.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5224795136 bytes 2026-02-19T20:21:40.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5251659776 bytes 2026-02-19T20:21:40.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5252269056 bytes 2026-02-19T20:21:40.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:21:40.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_19 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_19 snapshot 128M 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:21:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_19 128M 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:40.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:21:40.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:21:40.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:21:40.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:40.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_19 2026-02-19T20:21:40.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:21:40.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:40.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:21:40.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:40.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_19 snapshot 2026-02-19T20:21:40.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:40.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:40.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:21:40.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:40.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_19 snapshot 2026-02-19T20:21:41.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:21:41.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_19 2026-02-19T20:21:41.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_19': 2026-02-19T20:21:41.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:21:41.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:21:41.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:21:41.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1349b292076e 2026-02-19T20:21:41.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1349b292076e 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:40 2026 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:40 2026 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:40 2026 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 2bb7364a-f57b-450f-8771-0e4854fa0d34 2026-02-19T20:21:41.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:21:41.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:21:41.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_19 2026-02-19T20:21:41.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:41.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:41.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:21:41.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:21:41.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:21:41.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:21:41.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_19 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:21:41.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 44744704 bytes 2026-02-19T20:21:41.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 90583040 bytes 2026-02-19T20:21:41.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 172508160 bytes 2026-02-19T20:21:41.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 235567104 bytes 2026-02-19T20:21:42.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 295650304 bytes 2026-02-19T20:21:42.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 380226560 bytes 2026-02-19T20:21:42.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 421202944 bytes 2026-02-19T20:21:42.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 461308928 bytes 2026-02-19T20:21:42.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 478901248 bytes 2026-02-19T20:21:42.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 519168000 bytes 2026-02-19T20:21:42.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 577902592 bytes 2026-02-19T20:21:42.450 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:21:42.447+0000 7fab1dc3d640 -1 reset not still connected to 0x28f175f0 2026-02-19T20:21:42.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 641552384 bytes 2026-02-19T20:21:42.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 681883648 bytes 2026-02-19T20:21:42.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 715709440 bytes 2026-02-19T20:21:42.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 746491904 bytes 2026-02-19T20:21:42.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 778266624 bytes 2026-02-19T20:21:42.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 797307904 bytes 2026-02-19T20:21:42.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 836552704 bytes 2026-02-19T20:21:42.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 880106496 bytes 2026-02-19T20:21:42.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 903033856 bytes 2026-02-19T20:21:43.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 960585728 bytes 2026-02-19T20:21:43.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1051620352 bytes 2026-02-19T20:21:43.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1052211200 bytes 2026-02-19T20:21:43.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1077754880 bytes 2026-02-19T20:21:43.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1123244032 bytes 2026-02-19T20:21:43.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1175300096 bytes 2026-02-19T20:21:43.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1251414016 bytes 2026-02-19T20:21:43.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1288113152 bytes 2026-02-19T20:21:43.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1369552896 bytes 2026-02-19T20:21:43.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1408239616 bytes 2026-02-19T20:21:43.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1429176320 bytes 2026-02-19T20:21:43.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1453719552 bytes 2026-02-19T20:21:43.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1545102336 bytes 2026-02-19T20:21:43.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1608722432 bytes 2026-02-19T20:21:43.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1629752320 bytes 2026-02-19T20:21:43.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1654024192 bytes 2026-02-19T20:21:44.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1692741632 bytes 2026-02-19T20:21:44.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1693333504 bytes 2026-02-19T20:21:44.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1717894144 bytes 2026-02-19T20:21:44.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1734938624 bytes 2026-02-19T20:21:44.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1737176064 bytes 2026-02-19T20:21:44.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1793232896 bytes 2026-02-19T20:21:44.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1832240128 bytes 2026-02-19T20:21:44.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1871526912 bytes 2026-02-19T20:21:44.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1932248064 bytes 2026-02-19T20:21:44.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1939603456 bytes 2026-02-19T20:21:44.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2011548672 bytes 2026-02-19T20:21:44.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2089626624 bytes 2026-02-19T20:21:44.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2157494272 bytes 2026-02-19T20:21:45.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2208043008 bytes 2026-02-19T20:21:45.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2260272128 bytes 2026-02-19T20:21:45.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2281531392 bytes 2026-02-19T20:21:45.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2304239616 bytes 2026-02-19T20:21:45.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2343848960 bytes 2026-02-19T20:21:45.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2371623936 bytes 2026-02-19T20:21:45.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2426198016 bytes 2026-02-19T20:21:45.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2457247744 bytes 2026-02-19T20:21:45.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2474329088 bytes 2026-02-19T20:21:45.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2537605120 bytes 2026-02-19T20:21:45.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2559853568 bytes 2026-02-19T20:21:45.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2560483328 bytes 2026-02-19T20:21:45.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2596130816 bytes 2026-02-19T20:21:45.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2656852992 bytes 2026-02-19T20:21:45.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2731676672 bytes 2026-02-19T20:21:45.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2788212736 bytes 2026-02-19T20:21:45.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2809556992 bytes 2026-02-19T20:21:46.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2854695936 bytes 2026-02-19T20:21:46.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2930026496 bytes 2026-02-19T20:21:46.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2978350080 bytes 2026-02-19T20:21:46.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3032390656 bytes 2026-02-19T20:21:46.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3143663616 bytes 2026-02-19T20:21:46.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3190481920 bytes 2026-02-19T20:21:46.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3223673856 bytes 2026-02-19T20:21:46.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3267389440 bytes 2026-02-19T20:21:46.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3284483072 bytes 2026-02-19T20:21:46.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3338450944 bytes 2026-02-19T20:21:46.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3376461824 bytes 2026-02-19T20:21:46.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3404175360 bytes 2026-02-19T20:21:46.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3428170752 bytes 2026-02-19T20:21:46.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3482518528 bytes 2026-02-19T20:21:46.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3503295488 bytes 2026-02-19T20:21:47.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3542846464 bytes 2026-02-19T20:21:47.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3563241472 bytes 2026-02-19T20:21:47.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3563844608 bytes 2026-02-19T20:21:47.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3642618880 bytes 2026-02-19T20:21:47.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3707155456 bytes 2026-02-19T20:21:47.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3716928512 bytes 2026-02-19T20:21:47.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3747447808 bytes 2026-02-19T20:21:47.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3767888896 bytes 2026-02-19T20:21:47.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3832541184 bytes 2026-02-19T20:21:47.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3834930176 bytes 2026-02-19T20:21:47.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3875613696 bytes 2026-02-19T20:21:47.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3902462976 bytes 2026-02-19T20:21:47.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3961244672 bytes 2026-02-19T20:21:47.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4017473536 bytes 2026-02-19T20:21:47.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4107547648 bytes 2026-02-19T20:21:47.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4161346560 bytes 2026-02-19T20:21:47.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4218500096 bytes 2026-02-19T20:21:48.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4281597952 bytes 2026-02-19T20:21:48.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4330320896 bytes 2026-02-19T20:21:48.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4414740480 bytes 2026-02-19T20:21:48.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4465668096 bytes 2026-02-19T20:21:48.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4512158720 bytes 2026-02-19T20:21:48.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4578758656 bytes 2026-02-19T20:21:48.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4659082240 bytes 2026-02-19T20:21:48.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4694307840 bytes 2026-02-19T20:21:48.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4760614912 bytes 2026-02-19T20:21:48.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4811015168 bytes 2026-02-19T20:21:48.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4896333824 bytes 2026-02-19T20:21:48.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4929499136 bytes 2026-02-19T20:21:48.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4954001408 bytes 2026-02-19T20:21:48.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4989544448 bytes 2026-02-19T20:21:49.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5037587456 bytes 2026-02-19T20:21:49.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5057136640 bytes 2026-02-19T20:21:49.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5099368448 bytes 2026-02-19T20:21:49.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5131387904 bytes 2026-02-19T20:21:49.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5203744768 bytes 2026-02-19T20:21:49.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5252369408 bytes 2026-02-19T20:21:49.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5297496064 bytes 2026-02-19T20:21:49.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5372500992 bytes 2026-02-19T20:21:49.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5420751872 bytes 2026-02-19T20:21:49.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5455131648 bytes 2026-02-19T20:21:49.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5505452032 bytes 2026-02-19T20:21:49.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5562754048 bytes 2026-02-19T20:21:49.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5648872448 bytes 2026-02-19T20:21:49.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5679441920 bytes 2026-02-19T20:21:49.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5700173824 bytes 2026-02-19T20:21:50.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5717017600 bytes 2026-02-19T20:21:50.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5717633024 bytes 2026-02-19T20:21:50.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5770768384 bytes 2026-02-19T20:21:50.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5820562432 bytes 2026-02-19T20:21:50.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5877583872 bytes 2026-02-19T20:21:50.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5878198272 bytes 2026-02-19T20:21:50.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5952956416 bytes 2026-02-19T20:21:50.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5984958464 bytes 2026-02-19T20:21:50.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6029767680 bytes 2026-02-19T20:21:50.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6038562816 bytes 2026-02-19T20:21:50.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6064829440 bytes 2026-02-19T20:21:50.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6099653632 bytes 2026-02-19T20:21:50.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6127255552 bytes 2026-02-19T20:21:50.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6174566400 bytes 2026-02-19T20:21:50.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6289604608 bytes 2026-02-19T20:21:50.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6342234112 bytes 2026-02-19T20:21:51.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6394049536 bytes 2026-02-19T20:21:51.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6450213888 bytes 2026-02-19T20:21:51.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6493073408 bytes 2026-02-19T20:21:51.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6591906816 bytes 2026-02-19T20:21:51.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6659796992 bytes 2026-02-19T20:21:51.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6713267200 bytes 2026-02-19T20:21:51.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6744677376 bytes 2026-02-19T20:21:51.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6788259840 bytes 2026-02-19T20:21:51.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6890600448 bytes 2026-02-19T20:21:51.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6927053824 bytes 2026-02-19T20:21:51.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6970577920 bytes 2026-02-19T20:21:51.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6974684160 bytes 2026-02-19T20:21:51.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7009907712 bytes 2026-02-19T20:21:51.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7112962048 bytes 2026-02-19T20:21:52.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7142533120 bytes 2026-02-19T20:21:52.058 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4955540050189231) > weight (0.25) 2026-02-19T20:21:52.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2645811831009035) > weight (0.25) 2026-02-19T20:21:52.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3577456083827174) > weight (0.25) 2026-02-19T20:21:52.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.45216596994437963) > weight (0.25) 2026-02-19T20:21:52.059 INFO:tasks.rbd_mirror_thrash:waiting for 118.00547524028164 secs before thrashing 2026-02-19T20:21:52.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7233722368 bytes 2026-02-19T20:21:52.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7280785408 bytes 2026-02-19T20:21:52.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7350277120 bytes 2026-02-19T20:21:52.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7405964288 bytes 2026-02-19T20:21:52.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7423218688 bytes 2026-02-19T20:21:52.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7509312512 bytes 2026-02-19T20:21:52.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7548979200 bytes 2026-02-19T20:21:52.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7565654016 bytes 2026-02-19T20:21:52.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7632943104 bytes 2026-02-19T20:21:52.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7681005568 bytes 2026-02-19T20:21:52.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7732629504 bytes 2026-02-19T20:21:52.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7763690496 bytes 2026-02-19T20:21:52.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7811657728 bytes 2026-02-19T20:21:53.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7884314624 bytes 2026-02-19T20:21:53.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7921633280 bytes 2026-02-19T20:21:53.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7922222080 bytes 2026-02-19T20:21:53.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7982061568 bytes 2026-02-19T20:21:53.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 8007504896 bytes 2026-02-19T20:21:53.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 8059288576 bytes 2026-02-19T20:21:53.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 8144951296 bytes 2026-02-19T20:21:53.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 8202144768 bytes 2026-02-19T20:21:53.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 8231814144 bytes 2026-02-19T20:21:53.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 8256047104 bytes 2026-02-19T20:21:53.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:21:53.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:21:53.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:21:53.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:53.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:21:53.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_20 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_20 snapshot 128M 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_20 128M 2026-02-19T20:21:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:21:53.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_20 2026-02-19T20:21:53.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:21:53.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:53.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:21:53.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:21:53.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_20 snapshot 2026-02-19T20:21:53.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:53.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:53.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:21:53.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:21:53.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_20 snapshot 2026-02-19T20:21:54.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:21:54.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_20 2026-02-19T20:21:54.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_20': 2026-02-19T20:21:54.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:21:54.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:21:54.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:21:54.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1355ef02bd83 2026-02-19T20:21:54.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1355ef02bd83 2026-02-19T20:21:54.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:21:54.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:21:54.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:21:54.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:21:54.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:53 2026 2026-02-19T20:21:54.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:53 2026 2026-02-19T20:21:54.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:53 2026 2026-02-19T20:21:54.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:21:54.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:21:54.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 06be59c7-13b7-4ca9-8bf1-ac1f92b09a6e 2026-02-19T20:21:54.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:21:54.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:21:54.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_20 2026-02-19T20:21:54.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:21:54.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:21:54.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:21:54.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:21:54.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:21:54.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:21:54.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_20 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:21:54.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 85484544 bytes 2026-02-19T20:21:54.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 174106624 bytes 2026-02-19T20:21:54.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 191885312 bytes 2026-02-19T20:21:54.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 227745792 bytes 2026-02-19T20:21:54.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 228338688 bytes 2026-02-19T20:21:54.960 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8990751422621286) > weight (0.25) 2026-02-19T20:21:54.960 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9019180474222608) > weight (0.25) 2026-02-19T20:21:54.961 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9161990329440481) > weight (0.25) 2026-02-19T20:21:54.961 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.853999458728236) > weight (0.25) 2026-02-19T20:21:54.961 INFO:tasks.rbd_mirror_thrash:waiting for 103.11224076653858 secs before thrashing 2026-02-19T20:21:54.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 276734976 bytes 2026-02-19T20:21:55.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 305752064 bytes 2026-02-19T20:21:55.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 382106624 bytes 2026-02-19T20:21:55.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 442191872 bytes 2026-02-19T20:21:55.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 475385856 bytes 2026-02-19T20:21:55.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 476875776 bytes 2026-02-19T20:21:55.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 535016448 bytes 2026-02-19T20:21:55.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 589345792 bytes 2026-02-19T20:21:55.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 609297408 bytes 2026-02-19T20:21:55.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 633691136 bytes 2026-02-19T20:21:55.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 667415552 bytes 2026-02-19T20:21:55.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 697214976 bytes 2026-02-19T20:21:55.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 722388992 bytes 2026-02-19T20:21:55.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 770802688 bytes 2026-02-19T20:21:55.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 792788992 bytes 2026-02-19T20:21:55.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 875081728 bytes 2026-02-19T20:21:55.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 889700352 bytes 2026-02-19T20:21:55.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 946414592 bytes 2026-02-19T20:21:55.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1025335296 bytes 2026-02-19T20:21:56.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1063264256 bytes 2026-02-19T20:21:56.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1119831040 bytes 2026-02-19T20:21:56.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1175966720 bytes 2026-02-19T20:21:56.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1232046080 bytes 2026-02-19T20:21:56.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1255343104 bytes 2026-02-19T20:21:56.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1310467072 bytes 2026-02-19T20:21:56.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1342659584 bytes 2026-02-19T20:21:56.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1433571328 bytes 2026-02-19T20:21:56.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1516909568 bytes 2026-02-19T20:21:56.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1551803392 bytes 2026-02-19T20:21:56.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1604844544 bytes 2026-02-19T20:21:56.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1630621696 bytes 2026-02-19T20:21:56.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1650613248 bytes 2026-02-19T20:21:56.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1702405120 bytes 2026-02-19T20:21:56.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1725478912 bytes 2026-02-19T20:21:56.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1767100416 bytes 2026-02-19T20:21:57.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1792933888 bytes 2026-02-19T20:21:57.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1872711680 bytes 2026-02-19T20:21:57.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1892091904 bytes 2026-02-19T20:21:57.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1951803392 bytes 2026-02-19T20:21:57.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1965356032 bytes 2026-02-19T20:21:57.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2008127488 bytes 2026-02-19T20:21:57.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2033759232 bytes 2026-02-19T20:21:57.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2034376704 bytes 2026-02-19T20:21:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2052082688 bytes 2026-02-19T20:21:57.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2121755648 bytes 2026-02-19T20:21:57.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2148663296 bytes 2026-02-19T20:21:57.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2186173440 bytes 2026-02-19T20:21:57.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2262737920 bytes 2026-02-19T20:21:57.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2319759360 bytes 2026-02-19T20:21:57.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2362568704 bytes 2026-02-19T20:21:58.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2412419072 bytes 2026-02-19T20:21:58.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2451261440 bytes 2026-02-19T20:21:58.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2500220928 bytes 2026-02-19T20:21:58.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2562327552 bytes 2026-02-19T20:21:58.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2590651392 bytes 2026-02-19T20:21:58.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2633678848 bytes 2026-02-19T20:21:58.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2650022912 bytes 2026-02-19T20:21:58.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2705830912 bytes 2026-02-19T20:21:58.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2724720640 bytes 2026-02-19T20:21:58.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2749595648 bytes 2026-02-19T20:21:58.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2774571008 bytes 2026-02-19T20:21:58.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2831177728 bytes 2026-02-19T20:21:58.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2880543744 bytes 2026-02-19T20:21:58.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2944689152 bytes 2026-02-19T20:21:58.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3000133632 bytes 2026-02-19T20:21:58.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3055351808 bytes 2026-02-19T20:21:58.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3095302144 bytes 2026-02-19T20:21:59.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3133023232 bytes 2026-02-19T20:21:59.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3154254848 bytes 2026-02-19T20:21:59.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3164178432 bytes 2026-02-19T20:21:59.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3219616768 bytes 2026-02-19T20:21:59.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3291452416 bytes 2026-02-19T20:21:59.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3352243200 bytes 2026-02-19T20:21:59.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3393978368 bytes 2026-02-19T20:21:59.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3415741440 bytes 2026-02-19T20:21:59.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3426370560 bytes 2026-02-19T20:21:59.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3542496256 bytes 2026-02-19T20:21:59.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3586391040 bytes 2026-02-19T20:21:59.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3625024512 bytes 2026-02-19T20:21:59.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3644132352 bytes 2026-02-19T20:21:59.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3699961856 bytes 2026-02-19T20:21:59.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3700558848 bytes 2026-02-19T20:21:59.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3760942080 bytes 2026-02-19T20:21:59.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3800317952 bytes 2026-02-19T20:21:59.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3837039616 bytes 2026-02-19T20:22:00.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3848438784 bytes 2026-02-19T20:22:00.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3926908928 bytes 2026-02-19T20:22:00.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3951986688 bytes 2026-02-19T20:22:00.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3976388608 bytes 2026-02-19T20:22:00.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4002764800 bytes 2026-02-19T20:22:00.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4073744384 bytes 2026-02-19T20:22:00.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4107998208 bytes 2026-02-19T20:22:00.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4137026560 bytes 2026-02-19T20:22:00.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4143248384 bytes 2026-02-19T20:22:00.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4230693888 bytes 2026-02-19T20:22:00.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4306094080 bytes 2026-02-19T20:22:00.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4323152896 bytes 2026-02-19T20:22:00.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4339907584 bytes 2026-02-19T20:22:00.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4364694528 bytes 2026-02-19T20:22:00.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4415306752 bytes 2026-02-19T20:22:00.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4511299584 bytes 2026-02-19T20:22:00.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4589817856 bytes 2026-02-19T20:22:01.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4640714752 bytes 2026-02-19T20:22:01.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4681541632 bytes 2026-02-19T20:22:01.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4704772096 bytes 2026-02-19T20:22:01.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4716560384 bytes 2026-02-19T20:22:01.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4742789120 bytes 2026-02-19T20:22:01.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4761350144 bytes 2026-02-19T20:22:01.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4812858368 bytes 2026-02-19T20:22:01.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4874151936 bytes 2026-02-19T20:22:01.439 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4892046336 bytes 2026-02-19T20:22:01.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4913807360 bytes 2026-02-19T20:22:01.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4973892608 bytes 2026-02-19T20:22:01.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5021805568 bytes 2026-02-19T20:22:01.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5040678912 bytes 2026-02-19T20:22:01.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5091204096 bytes 2026-02-19T20:22:01.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5113823232 bytes 2026-02-19T20:22:01.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5146614784 bytes 2026-02-19T20:22:01.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5181700096 bytes 2026-02-19T20:22:01.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5182317568 bytes 2026-02-19T20:22:01.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5218594816 bytes 2026-02-19T20:22:01.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5223184384 bytes 2026-02-19T20:22:02.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5271263232 bytes 2026-02-19T20:22:02.103 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5319969792 bytes 2026-02-19T20:22:02.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5366535168 bytes 2026-02-19T20:22:02.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5378409472 bytes 2026-02-19T20:22:02.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5478195200 bytes 2026-02-19T20:22:02.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5514333184 bytes 2026-02-19T20:22:02.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5575116800 bytes 2026-02-19T20:22:02.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5653379072 bytes 2026-02-19T20:22:02.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5698021376 bytes 2026-02-19T20:22:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5751057408 bytes 2026-02-19T20:22:02.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5788675072 bytes 2026-02-19T20:22:02.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5845913600 bytes 2026-02-19T20:22:02.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5884242944 bytes 2026-02-19T20:22:02.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5946304512 bytes 2026-02-19T20:22:02.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6016818176 bytes 2026-02-19T20:22:03.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6059335680 bytes 2026-02-19T20:22:03.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6070583296 bytes 2026-02-19T20:22:03.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6109961216 bytes 2026-02-19T20:22:03.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6179373056 bytes 2026-02-19T20:22:03.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6189285376 bytes 2026-02-19T20:22:03.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6262220800 bytes 2026-02-19T20:22:03.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6317083648 bytes 2026-02-19T20:22:03.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6342067200 bytes 2026-02-19T20:22:03.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6352398336 bytes 2026-02-19T20:22:03.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6411196416 bytes 2026-02-19T20:22:03.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6458083328 bytes 2026-02-19T20:22:03.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6501392384 bytes 2026-02-19T20:22:03.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6552120320 bytes 2026-02-19T20:22:03.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6593862656 bytes 2026-02-19T20:22:04.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6594463744 bytes 2026-02-19T20:22:04.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6646934528 bytes 2026-02-19T20:22:04.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6687480832 bytes 2026-02-19T20:22:04.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6722332672 bytes 2026-02-19T20:22:04.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6852410368 bytes 2026-02-19T20:22:04.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6918040576 bytes 2026-02-19T20:22:04.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6918643712 bytes 2026-02-19T20:22:04.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6919267328 bytes 2026-02-19T20:22:04.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6933555200 bytes 2026-02-19T20:22:04.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7042233344 bytes 2026-02-19T20:22:04.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:22:04.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_21 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_21 snapshot 128M 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:04.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_21 128M 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:22:04.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:22:04.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:22:04.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:04.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_21 2026-02-19T20:22:04.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:22:04.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:04.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:22:04.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:04.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_21 snapshot 2026-02-19T20:22:04.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:04.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:04.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:22:04.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:04.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_21 snapshot 2026-02-19T20:22:05.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:22:05.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_21 2026-02-19T20:22:05.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_21': 2026-02-19T20:22:05.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:22:05.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:22:05.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:22:05.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 13617a740151 2026-02-19T20:22:05.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.13617a740151 2026-02-19T20:22:05.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:04 2026 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:04 2026 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:04 2026 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 6a7c4609-46c1-4458-b2e2-2fc6d6f26028 2026-02-19T20:22:05.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:22:05.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:22:05.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_21 2026-02-19T20:22:05.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:05.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:05.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:22:05.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:22:05.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=8 2026-02-19T20:22:05.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:22:05.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_21 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:22:05.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 32771072 bytes 2026-02-19T20:22:05.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 74507264 bytes 2026-02-19T20:22:05.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 177508352 bytes 2026-02-19T20:22:05.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 183486464 bytes 2026-02-19T20:22:05.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 227741696 bytes 2026-02-19T20:22:06.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 264760320 bytes 2026-02-19T20:22:06.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 347654144 bytes 2026-02-19T20:22:06.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 388055040 bytes 2026-02-19T20:22:06.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 453820416 bytes 2026-02-19T20:22:06.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 488834048 bytes 2026-02-19T20:22:06.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 540718080 bytes 2026-02-19T20:22:06.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 572540928 bytes 2026-02-19T20:22:06.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 590316544 bytes 2026-02-19T20:22:06.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 636614656 bytes 2026-02-19T20:22:06.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 667327488 bytes 2026-02-19T20:22:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 699075584 bytes 2026-02-19T20:22:06.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 736346112 bytes 2026-02-19T20:22:06.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 763787264 bytes 2026-02-19T20:22:06.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 812937216 bytes 2026-02-19T20:22:06.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 868679680 bytes 2026-02-19T20:22:06.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 889949184 bytes 2026-02-19T20:22:06.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 901995520 bytes 2026-02-19T20:22:07.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 981548032 bytes 2026-02-19T20:22:07.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1062046720 bytes 2026-02-19T20:22:07.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1102050304 bytes 2026-02-19T20:22:07.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1135510528 bytes 2026-02-19T20:22:07.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1214862336 bytes 2026-02-19T20:22:07.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1240887296 bytes 2026-02-19T20:22:07.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1284108288 bytes 2026-02-19T20:22:07.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1329274880 bytes 2026-02-19T20:22:07.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1398986752 bytes 2026-02-19T20:22:07.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1440563200 bytes 2026-02-19T20:22:07.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1507914752 bytes 2026-02-19T20:22:07.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1553019904 bytes 2026-02-19T20:22:07.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1585804288 bytes 2026-02-19T20:22:07.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1586418688 bytes 2026-02-19T20:22:07.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1637239808 bytes 2026-02-19T20:22:08.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1709438976 bytes 2026-02-19T20:22:08.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1771195392 bytes 2026-02-19T20:22:08.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1818723328 bytes 2026-02-19T20:22:08.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1833968640 bytes 2026-02-19T20:22:08.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1851006976 bytes 2026-02-19T20:22:08.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1886498816 bytes 2026-02-19T20:22:08.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1936081920 bytes 2026-02-19T20:22:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1965644800 bytes 2026-02-19T20:22:08.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1988764672 bytes 2026-02-19T20:22:08.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2030593024 bytes 2026-02-19T20:22:08.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2052820992 bytes 2026-02-19T20:22:08.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2139854848 bytes 2026-02-19T20:22:08.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2140467200 bytes 2026-02-19T20:22:08.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2186629120 bytes 2026-02-19T20:22:08.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2252876800 bytes 2026-02-19T20:22:08.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2277221376 bytes 2026-02-19T20:22:09.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2340861952 bytes 2026-02-19T20:22:09.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2366201856 bytes 2026-02-19T20:22:09.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2457322496 bytes 2026-02-19T20:22:09.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2481786880 bytes 2026-02-19T20:22:09.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2492319744 bytes 2026-02-19T20:22:09.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2619660288 bytes 2026-02-19T20:22:09.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2725404672 bytes 2026-02-19T20:22:09.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2824092672 bytes 2026-02-19T20:22:09.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2912989184 bytes 2026-02-19T20:22:09.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2935956480 bytes 2026-02-19T20:22:09.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2998703104 bytes 2026-02-19T20:22:09.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3055852544 bytes 2026-02-19T20:22:09.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3139617792 bytes 2026-02-19T20:22:09.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3140234240 bytes 2026-02-19T20:22:10.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3184830464 bytes 2026-02-19T20:22:10.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3217438720 bytes 2026-02-19T20:22:10.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3242610688 bytes 2026-02-19T20:22:10.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3275291648 bytes 2026-02-19T20:22:10.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3302111232 bytes 2026-02-19T20:22:10.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3324710912 bytes 2026-02-19T20:22:10.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3375298560 bytes 2026-02-19T20:22:10.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3411932160 bytes 2026-02-19T20:22:10.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3412543488 bytes 2026-02-19T20:22:10.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3432087552 bytes 2026-02-19T20:22:10.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3484430336 bytes 2026-02-19T20:22:10.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3510043648 bytes 2026-02-19T20:22:10.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3538277376 bytes 2026-02-19T20:22:10.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3574806528 bytes 2026-02-19T20:22:10.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3584499712 bytes 2026-02-19T20:22:10.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3646371840 bytes 2026-02-19T20:22:10.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3681447936 bytes 2026-02-19T20:22:10.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3695255552 bytes 2026-02-19T20:22:10.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3764491264 bytes 2026-02-19T20:22:11.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3821831168 bytes 2026-02-19T20:22:11.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3859174400 bytes 2026-02-19T20:22:11.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3874964480 bytes 2026-02-19T20:22:11.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3935698944 bytes 2026-02-19T20:22:11.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3974660096 bytes 2026-02-19T20:22:11.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4038965248 bytes 2026-02-19T20:22:11.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4059982848 bytes 2026-02-19T20:22:11.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4078205952 bytes 2026-02-19T20:22:11.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4126401536 bytes 2026-02-19T20:22:11.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4186343424 bytes 2026-02-19T20:22:11.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4258164736 bytes 2026-02-19T20:22:11.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4340425728 bytes 2026-02-19T20:22:11.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4385738752 bytes 2026-02-19T20:22:11.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4435458048 bytes 2026-02-19T20:22:11.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4448730112 bytes 2026-02-19T20:22:11.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4453341184 bytes 2026-02-19T20:22:12.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4490940416 bytes 2026-02-19T20:22:12.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4586487808 bytes 2026-02-19T20:22:12.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4650636288 bytes 2026-02-19T20:22:12.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4661392384 bytes 2026-02-19T20:22:12.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4688813056 bytes 2026-02-19T20:22:12.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4713074688 bytes 2026-02-19T20:22:12.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4784545792 bytes 2026-02-19T20:22:12.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4794473472 bytes 2026-02-19T20:22:12.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4810982400 bytes 2026-02-19T20:22:12.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4854405120 bytes 2026-02-19T20:22:12.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4943718400 bytes 2026-02-19T20:22:12.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4986873856 bytes 2026-02-19T20:22:12.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4993028096 bytes 2026-02-19T20:22:12.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5014182912 bytes 2026-02-19T20:22:12.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5128888320 bytes 2026-02-19T20:22:12.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5147264000 bytes 2026-02-19T20:22:13.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5215058944 bytes 2026-02-19T20:22:13.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5242820608 bytes 2026-02-19T20:22:13.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5275033600 bytes 2026-02-19T20:22:13.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5332904960 bytes 2026-02-19T20:22:13.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5405665280 bytes 2026-02-19T20:22:13.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5431616512 bytes 2026-02-19T20:22:13.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5475537920 bytes 2026-02-19T20:22:13.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5534005248 bytes 2026-02-19T20:22:13.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5534623744 bytes 2026-02-19T20:22:13.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5553676288 bytes 2026-02-19T20:22:13.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5558316032 bytes 2026-02-19T20:22:13.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:22:13.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:22:13.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:22:13.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_22 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_22 snapshot 128M 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_22 128M 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:22:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:22:13.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:22:13.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:13.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_22 2026-02-19T20:22:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:22:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:22:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_22 snapshot 2026-02-19T20:22:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:22:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_22 snapshot 2026-02-19T20:22:14.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:22:14.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_22 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_22': 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 136d321324a3 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.136d321324a3 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:13 2026 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:13 2026 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:13 2026 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: f9647a4e-5b33-4f6e-bf71-dcbe6192f2bc 2026-02-19T20:22:14.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:22:14.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:22:14.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_22 2026-02-19T20:22:14.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:14.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:14.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:22:14.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:22:14.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:22:14.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:22:14.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_22 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:22:14.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 20570112 bytes 2026-02-19T20:22:14.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 66058240 bytes 2026-02-19T20:22:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 98882560 bytes 2026-02-19T20:22:14.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 147850240 bytes 2026-02-19T20:22:14.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 155934720 bytes 2026-02-19T20:22:14.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 228904960 bytes 2026-02-19T20:22:14.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 244559872 bytes 2026-02-19T20:22:15.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 322325504 bytes 2026-02-19T20:22:15.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 360426496 bytes 2026-02-19T20:22:15.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 375820288 bytes 2026-02-19T20:22:15.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 433201152 bytes 2026-02-19T20:22:15.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 440019968 bytes 2026-02-19T20:22:15.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 454150144 bytes 2026-02-19T20:22:15.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 503708672 bytes 2026-02-19T20:22:15.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 557932544 bytes 2026-02-19T20:22:15.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 598087680 bytes 2026-02-19T20:22:15.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 631590912 bytes 2026-02-19T20:22:15.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 744852480 bytes 2026-02-19T20:22:15.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 792151040 bytes 2026-02-19T20:22:15.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 802209792 bytes 2026-02-19T20:22:15.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 869820416 bytes 2026-02-19T20:22:15.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 918431744 bytes 2026-02-19T20:22:15.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 939938816 bytes 2026-02-19T20:22:16.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 962451456 bytes 2026-02-19T20:22:16.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1009768448 bytes 2026-02-19T20:22:16.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1052908544 bytes 2026-02-19T20:22:16.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1109493760 bytes 2026-02-19T20:22:16.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1147979776 bytes 2026-02-19T20:22:16.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1192580096 bytes 2026-02-19T20:22:16.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1233445888 bytes 2026-02-19T20:22:16.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1255510016 bytes 2026-02-19T20:22:16.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1276481536 bytes 2026-02-19T20:22:16.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1277087744 bytes 2026-02-19T20:22:16.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1340194816 bytes 2026-02-19T20:22:16.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1369059328 bytes 2026-02-19T20:22:16.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1452126208 bytes 2026-02-19T20:22:16.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1519416320 bytes 2026-02-19T20:22:16.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1590339584 bytes 2026-02-19T20:22:16.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1609576448 bytes 2026-02-19T20:22:16.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1620226048 bytes 2026-02-19T20:22:17.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1651604480 bytes 2026-02-19T20:22:17.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1681346560 bytes 2026-02-19T20:22:17.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1704696832 bytes 2026-02-19T20:22:17.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1730563072 bytes 2026-02-19T20:22:17.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1756263424 bytes 2026-02-19T20:22:17.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1790915584 bytes 2026-02-19T20:22:17.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1830834176 bytes 2026-02-19T20:22:17.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1860076544 bytes 2026-02-19T20:22:17.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1926344704 bytes 2026-02-19T20:22:17.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1962610688 bytes 2026-02-19T20:22:17.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1979346944 bytes 2026-02-19T20:22:17.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2000720896 bytes 2026-02-19T20:22:17.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2069474304 bytes 2026-02-19T20:22:17.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2138442752 bytes 2026-02-19T20:22:17.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2144662528 bytes 2026-02-19T20:22:17.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2171524096 bytes 2026-02-19T20:22:17.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2241872896 bytes 2026-02-19T20:22:18.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2295026688 bytes 2026-02-19T20:22:18.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2343290880 bytes 2026-02-19T20:22:18.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2383613952 bytes 2026-02-19T20:22:18.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2398954496 bytes 2026-02-19T20:22:18.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2480008192 bytes 2026-02-19T20:22:18.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2534391808 bytes 2026-02-19T20:22:18.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2568466432 bytes 2026-02-19T20:22:18.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2641200128 bytes 2026-02-19T20:22:18.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2662806528 bytes 2026-02-19T20:22:18.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2728733696 bytes 2026-02-19T20:22:18.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2761904128 bytes 2026-02-19T20:22:18.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2793889792 bytes 2026-02-19T20:22:18.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2863668224 bytes 2026-02-19T20:22:18.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2910558208 bytes 2026-02-19T20:22:18.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2972475392 bytes 2026-02-19T20:22:18.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3003682816 bytes 2026-02-19T20:22:19.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3042723840 bytes 2026-02-19T20:22:19.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3125104640 bytes 2026-02-19T20:22:19.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3175234560 bytes 2026-02-19T20:22:19.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3223595008 bytes 2026-02-19T20:22:19.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3263205376 bytes 2026-02-19T20:22:19.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3278761984 bytes 2026-02-19T20:22:19.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3295881216 bytes 2026-02-19T20:22:19.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3356918784 bytes 2026-02-19T20:22:19.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3391851520 bytes 2026-02-19T20:22:19.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3434361856 bytes 2026-02-19T20:22:19.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3489586176 bytes 2026-02-19T20:22:19.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3518294016 bytes 2026-02-19T20:22:19.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3552155648 bytes 2026-02-19T20:22:19.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3583117312 bytes 2026-02-19T20:22:20.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3604542464 bytes 2026-02-19T20:22:20.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3605173248 bytes 2026-02-19T20:22:20.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3667397632 bytes 2026-02-19T20:22:20.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3693259776 bytes 2026-02-19T20:22:20.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3737366528 bytes 2026-02-19T20:22:20.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3796458496 bytes 2026-02-19T20:22:20.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3859469312 bytes 2026-02-19T20:22:20.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3886876672 bytes 2026-02-19T20:22:20.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3930715136 bytes 2026-02-19T20:22:20.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3932014592 bytes 2026-02-19T20:22:20.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3956113408 bytes 2026-02-19T20:22:20.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3996899328 bytes 2026-02-19T20:22:20.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4057155584 bytes 2026-02-19T20:22:20.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4057787392 bytes 2026-02-19T20:22:20.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4091652096 bytes 2026-02-19T20:22:20.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4155410432 bytes 2026-02-19T20:22:20.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4196050944 bytes 2026-02-19T20:22:20.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4233467904 bytes 2026-02-19T20:22:21.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4299880448 bytes 2026-02-19T20:22:21.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4316340224 bytes 2026-02-19T20:22:21.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4376994816 bytes 2026-02-19T20:22:21.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4390633472 bytes 2026-02-19T20:22:21.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4436554752 bytes 2026-02-19T20:22:21.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4474715136 bytes 2026-02-19T20:22:21.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4523732992 bytes 2026-02-19T20:22:21.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4578983936 bytes 2026-02-19T20:22:21.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4596078592 bytes 2026-02-19T20:22:21.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4638821376 bytes 2026-02-19T20:22:21.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4657780736 bytes 2026-02-19T20:22:21.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4724718592 bytes 2026-02-19T20:22:21.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4729335808 bytes 2026-02-19T20:22:21.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4766945280 bytes 2026-02-19T20:22:21.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4817735680 bytes 2026-02-19T20:22:21.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4821565440 bytes 2026-02-19T20:22:21.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4841911296 bytes 2026-02-19T20:22:21.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4910272512 bytes 2026-02-19T20:22:22.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4960078848 bytes 2026-02-19T20:22:22.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 4995809280 bytes 2026-02-19T20:22:22.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5069153280 bytes 2026-02-19T20:22:22.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5122888704 bytes 2026-02-19T20:22:22.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5164442624 bytes 2026-02-19T20:22:22.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5229087744 bytes 2026-02-19T20:22:22.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5254328320 bytes 2026-02-19T20:22:22.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5309578240 bytes 2026-02-19T20:22:22.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5319171072 bytes 2026-02-19T20:22:22.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5340829696 bytes 2026-02-19T20:22:22.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5428918272 bytes 2026-02-19T20:22:22.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5467260928 bytes 2026-02-19T20:22:22.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5564106752 bytes 2026-02-19T20:22:22.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5581650944 bytes 2026-02-19T20:22:22.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5607347200 bytes 2026-02-19T20:22:22.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5622764544 bytes 2026-02-19T20:22:23.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5715894272 bytes 2026-02-19T20:22:23.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5752407040 bytes 2026-02-19T20:22:23.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5794809856 bytes 2026-02-19T20:22:23.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5811236864 bytes 2026-02-19T20:22:23.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5849368576 bytes 2026-02-19T20:22:23.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5933508608 bytes 2026-02-19T20:22:23.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5945484288 bytes 2026-02-19T20:22:23.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5962931200 bytes 2026-02-19T20:22:23.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5963533312 bytes 2026-02-19T20:22:23.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6002989056 bytes 2026-02-19T20:22:23.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6018373632 bytes 2026-02-19T20:22:23.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6045062144 bytes 2026-02-19T20:22:23.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6081841152 bytes 2026-02-19T20:22:23.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6134226944 bytes 2026-02-19T20:22:23.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6153698304 bytes 2026-02-19T20:22:23.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6194960384 bytes 2026-02-19T20:22:23.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6268193792 bytes 2026-02-19T20:22:24.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6315916288 bytes 2026-02-19T20:22:24.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6354499584 bytes 2026-02-19T20:22:24.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6406995968 bytes 2026-02-19T20:22:24.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6425888768 bytes 2026-02-19T20:22:24.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6505901056 bytes 2026-02-19T20:22:24.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6598059008 bytes 2026-02-19T20:22:24.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6659213312 bytes 2026-02-19T20:22:24.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6670107648 bytes 2026-02-19T20:22:24.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6698663936 bytes 2026-02-19T20:22:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6826332160 bytes 2026-02-19T20:22:24.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6903213056 bytes 2026-02-19T20:22:24.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6928841728 bytes 2026-02-19T20:22:24.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6961971200 bytes 2026-02-19T20:22:24.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6983072768 bytes 2026-02-19T20:22:24.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7029730304 bytes 2026-02-19T20:22:25.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7045267456 bytes 2026-02-19T20:22:25.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7061456896 bytes 2026-02-19T20:22:25.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7100736512 bytes 2026-02-19T20:22:25.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7175404544 bytes 2026-02-19T20:22:25.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7260548096 bytes 2026-02-19T20:22:25.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7327524864 bytes 2026-02-19T20:22:25.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7397364736 bytes 2026-02-19T20:22:25.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7435634688 bytes 2026-02-19T20:22:25.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7479946240 bytes 2026-02-19T20:22:25.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7480558592 bytes 2026-02-19T20:22:25.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7532457984 bytes 2026-02-19T20:22:25.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7548566528 bytes 2026-02-19T20:22:25.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7604605952 bytes 2026-02-19T20:22:25.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7620919296 bytes 2026-02-19T20:22:25.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7665387520 bytes 2026-02-19T20:22:25.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7706151936 bytes 2026-02-19T20:22:25.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7706761216 bytes 2026-02-19T20:22:26.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7761223680 bytes 2026-02-19T20:22:26.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7842694144 bytes 2026-02-19T20:22:26.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7901519872 bytes 2026-02-19T20:22:26.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7938670592 bytes 2026-02-19T20:22:26.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7999016960 bytes 2026-02-19T20:22:26.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8061051904 bytes 2026-02-19T20:22:26.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8108931072 bytes 2026-02-19T20:22:26.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8117666816 bytes 2026-02-19T20:22:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8154826752 bytes 2026-02-19T20:22:26.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_23 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_23 snapshot 128M 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:26.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_23 128M 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:22:26.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:26.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:22:26.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:22:26.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:22:26.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:26.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_23 2026-02-19T20:22:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:22:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:22:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_23 snapshot 2026-02-19T20:22:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:26.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:26.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:22:26.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:26.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_23 snapshot 2026-02-19T20:22:27.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:22:27.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_23 2026-02-19T20:22:27.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_23': 2026-02-19T20:22:27.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:22:27.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:22:27.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:22:27.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1379f3481f56 2026-02-19T20:22:27.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1379f3481f56 2026-02-19T20:22:27.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:22:27.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:22:27.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:22:27.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:22:27.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:26 2026 2026-02-19T20:22:27.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:26 2026 2026-02-19T20:22:27.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:26 2026 2026-02-19T20:22:27.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:22:27.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:22:27.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 8b84cd27-943f-470d-8e23-786f49d70f13 2026-02-19T20:22:27.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:22:27.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:22:27.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_23 2026-02-19T20:22:27.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:27.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:27.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:22:27.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:22:27.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=11 2026-02-19T20:22:27.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:22:27.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_23 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:22:27.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 10710016 bytes 2026-02-19T20:22:27.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 23702528 bytes 2026-02-19T20:22:27.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 54091776 bytes 2026-02-19T20:22:27.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 103145472 bytes 2026-02-19T20:22:27.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 193466368 bytes 2026-02-19T20:22:27.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 225524736 bytes 2026-02-19T20:22:28.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 256891904 bytes 2026-02-19T20:22:28.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 270694400 bytes 2026-02-19T20:22:28.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 310037504 bytes 2026-02-19T20:22:28.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 331583488 bytes 2026-02-19T20:22:28.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 368108544 bytes 2026-02-19T20:22:28.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 421804032 bytes 2026-02-19T20:22:28.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 487924736 bytes 2026-02-19T20:22:28.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 524186624 bytes 2026-02-19T20:22:28.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 628055040 bytes 2026-02-19T20:22:28.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 687763456 bytes 2026-02-19T20:22:28.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 692942848 bytes 2026-02-19T20:22:28.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 702626816 bytes 2026-02-19T20:22:28.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 849892352 bytes 2026-02-19T20:22:28.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 883573760 bytes 2026-02-19T20:22:28.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 935918592 bytes 2026-02-19T20:22:29.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 973868032 bytes 2026-02-19T20:22:29.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1012889600 bytes 2026-02-19T20:22:29.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1055069184 bytes 2026-02-19T20:22:29.196 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1070918656 bytes 2026-02-19T20:22:29.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1072190464 bytes 2026-02-19T20:22:29.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1126417408 bytes 2026-02-19T20:22:29.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1127056384 bytes 2026-02-19T20:22:29.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1193489408 bytes 2026-02-19T20:22:29.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1212006400 bytes 2026-02-19T20:22:29.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1253059584 bytes 2026-02-19T20:22:29.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1274735616 bytes 2026-02-19T20:22:29.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1313040384 bytes 2026-02-19T20:22:29.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1350362112 bytes 2026-02-19T20:22:29.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1401044992 bytes 2026-02-19T20:22:29.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1425506304 bytes 2026-02-19T20:22:29.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1473061888 bytes 2026-02-19T20:22:29.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1495894016 bytes 2026-02-19T20:22:29.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1533835264 bytes 2026-02-19T20:22:30.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1590805504 bytes 2026-02-19T20:22:30.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1662297088 bytes 2026-02-19T20:22:30.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1685964800 bytes 2026-02-19T20:22:30.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1702838272 bytes 2026-02-19T20:22:30.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1732855808 bytes 2026-02-19T20:22:30.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1764807680 bytes 2026-02-19T20:22:30.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1795765248 bytes 2026-02-19T20:22:30.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1841849344 bytes 2026-02-19T20:22:30.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1883743232 bytes 2026-02-19T20:22:30.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1890359296 bytes 2026-02-19T20:22:30.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1953872896 bytes 2026-02-19T20:22:30.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2026411008 bytes 2026-02-19T20:22:30.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2047551488 bytes 2026-02-19T20:22:30.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2162380800 bytes 2026-02-19T20:22:30.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2214393856 bytes 2026-02-19T20:22:30.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2214983680 bytes 2026-02-19T20:22:31.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2306908160 bytes 2026-02-19T20:22:31.103 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2348315648 bytes 2026-02-19T20:22:31.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2379383808 bytes 2026-02-19T20:22:31.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2426327040 bytes 2026-02-19T20:22:31.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2494923776 bytes 2026-02-19T20:22:31.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2581202944 bytes 2026-02-19T20:22:31.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2632262656 bytes 2026-02-19T20:22:31.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2667579392 bytes 2026-02-19T20:22:31.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2695455744 bytes 2026-02-19T20:22:31.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2759173120 bytes 2026-02-19T20:22:31.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2820504576 bytes 2026-02-19T20:22:31.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2838751232 bytes 2026-02-19T20:22:31.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2849555456 bytes 2026-02-19T20:22:31.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2923047936 bytes 2026-02-19T20:22:31.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2943107072 bytes 2026-02-19T20:22:32.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2983761920 bytes 2026-02-19T20:22:32.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3022105600 bytes 2026-02-19T20:22:32.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3120428032 bytes 2026-02-19T20:22:32.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3140705280 bytes 2026-02-19T20:22:32.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3195452416 bytes 2026-02-19T20:22:32.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3215073280 bytes 2026-02-19T20:22:32.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3299809280 bytes 2026-02-19T20:22:32.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3314291712 bytes 2026-02-19T20:22:32.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3357933568 bytes 2026-02-19T20:22:32.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3423560704 bytes 2026-02-19T20:22:32.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3434052608 bytes 2026-02-19T20:22:32.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3474304000 bytes 2026-02-19T20:22:32.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3523493888 bytes 2026-02-19T20:22:32.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3527405568 bytes 2026-02-19T20:22:32.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3563863040 bytes 2026-02-19T20:22:32.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3642941440 bytes 2026-02-19T20:22:32.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3657311232 bytes 2026-02-19T20:22:33.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3680559104 bytes 2026-02-19T20:22:33.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3712696320 bytes 2026-02-19T20:22:33.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3734512640 bytes 2026-02-19T20:22:33.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3824732160 bytes 2026-02-19T20:22:33.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3869878272 bytes 2026-02-19T20:22:33.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3894984704 bytes 2026-02-19T20:22:33.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3965073408 bytes 2026-02-19T20:22:33.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4011933696 bytes 2026-02-19T20:22:33.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4080408576 bytes 2026-02-19T20:22:33.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4105015296 bytes 2026-02-19T20:22:33.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4126465024 bytes 2026-02-19T20:22:33.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4137665536 bytes 2026-02-19T20:22:33.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4171550720 bytes 2026-02-19T20:22:33.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4221587456 bytes 2026-02-19T20:22:33.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4232050688 bytes 2026-02-19T20:22:33.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4247557120 bytes 2026-02-19T20:22:34.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4335984640 bytes 2026-02-19T20:22:34.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4336601088 bytes 2026-02-19T20:22:34.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4371424256 bytes 2026-02-19T20:22:34.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4399985664 bytes 2026-02-19T20:22:34.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4458446848 bytes 2026-02-19T20:22:34.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4507828224 bytes 2026-02-19T20:22:34.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4553136128 bytes 2026-02-19T20:22:34.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4635722752 bytes 2026-02-19T20:22:34.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4670360576 bytes 2026-02-19T20:22:34.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4700317696 bytes 2026-02-19T20:22:34.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4821177344 bytes 2026-02-19T20:22:34.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4867892224 bytes 2026-02-19T20:22:34.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4887473152 bytes 2026-02-19T20:22:34.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4945011712 bytes 2026-02-19T20:22:35.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5047710720 bytes 2026-02-19T20:22:35.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5102160896 bytes 2026-02-19T20:22:35.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5162059776 bytes 2026-02-19T20:22:35.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5203048448 bytes 2026-02-19T20:22:35.292 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5239291904 bytes 2026-02-19T20:22:35.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5307635712 bytes 2026-02-19T20:22:35.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5368901632 bytes 2026-02-19T20:22:35.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5414093824 bytes 2026-02-19T20:22:35.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5465724928 bytes 2026-02-19T20:22:35.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5490193408 bytes 2026-02-19T20:22:35.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5542425600 bytes 2026-02-19T20:22:35.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5579258880 bytes 2026-02-19T20:22:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5602667520 bytes 2026-02-19T20:22:35.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5636817920 bytes 2026-02-19T20:22:35.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5671723008 bytes 2026-02-19T20:22:35.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5694935040 bytes 2026-02-19T20:22:36.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5750605824 bytes 2026-02-19T20:22:36.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5808142336 bytes 2026-02-19T20:22:36.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5823824896 bytes 2026-02-19T20:22:36.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5848446976 bytes 2026-02-19T20:22:36.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5864374272 bytes 2026-02-19T20:22:36.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5883209728 bytes 2026-02-19T20:22:36.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5911203840 bytes 2026-02-19T20:22:36.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5958964224 bytes 2026-02-19T20:22:36.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6009327616 bytes 2026-02-19T20:22:36.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6032642048 bytes 2026-02-19T20:22:36.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6057653248 bytes 2026-02-19T20:22:36.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6081081344 bytes 2026-02-19T20:22:36.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6163960832 bytes 2026-02-19T20:22:36.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6239312896 bytes 2026-02-19T20:22:36.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6239928320 bytes 2026-02-19T20:22:36.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6252354560 bytes 2026-02-19T20:22:36.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6291164160 bytes 2026-02-19T20:22:37.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6303851520 bytes 2026-02-19T20:22:37.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6347235328 bytes 2026-02-19T20:22:37.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6455174144 bytes 2026-02-19T20:22:37.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6458525696 bytes 2026-02-19T20:22:37.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6489017344 bytes 2026-02-19T20:22:37.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6503754752 bytes 2026-02-19T20:22:37.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6564960256 bytes 2026-02-19T20:22:37.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6704913408 bytes 2026-02-19T20:22:37.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6723899392 bytes 2026-02-19T20:22:37.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6748230656 bytes 2026-02-19T20:22:37.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6749418496 bytes 2026-02-19T20:22:37.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6766919680 bytes 2026-02-19T20:22:37.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6788728832 bytes 2026-02-19T20:22:37.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6810499072 bytes 2026-02-19T20:22:37.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6839549952 bytes 2026-02-19T20:22:37.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6879878144 bytes 2026-02-19T20:22:37.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6899871744 bytes 2026-02-19T20:22:38.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6985285632 bytes 2026-02-19T20:22:38.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7052262400 bytes 2026-02-19T20:22:38.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7072325632 bytes 2026-02-19T20:22:38.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7104446464 bytes 2026-02-19T20:22:38.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7148209152 bytes 2026-02-19T20:22:38.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7197234176 bytes 2026-02-19T20:22:38.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7273697280 bytes 2026-02-19T20:22:38.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7302974464 bytes 2026-02-19T20:22:38.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7336157184 bytes 2026-02-19T20:22:38.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_24 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_24 snapshot 128M 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:38.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_24 128M 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:22:38.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:38.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:22:38.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:22:38.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:22:38.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:38.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_24 2026-02-19T20:22:38.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:22:38.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:38.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:22:38.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:38.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_24 snapshot 2026-02-19T20:22:38.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:38.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:38.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:22:38.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:38.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_24 snapshot 2026-02-19T20:22:39.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:22:39.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_24 2026-02-19T20:22:39.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_24': 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 138551e483f7 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.138551e483f7 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:22:39.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:38 2026 2026-02-19T20:22:39.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:38 2026 2026-02-19T20:22:39.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:38 2026 2026-02-19T20:22:39.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:22:39.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:22:39.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 79b55a6a-adb4-4d23-a92b-d5cbf63850f4 2026-02-19T20:22:39.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:22:39.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:22:39.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_24 2026-02-19T20:22:39.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:39.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:39.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:22:39.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:22:39.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=7 2026-02-19T20:22:39.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:22:39.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_24 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:22:39.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 7300096 bytes 2026-02-19T20:22:39.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 45329408 bytes 2026-02-19T20:22:39.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 84485120 bytes 2026-02-19T20:22:39.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 133464064 bytes 2026-02-19T20:22:39.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 159931392 bytes 2026-02-19T20:22:39.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 173027328 bytes 2026-02-19T20:22:40.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 266625024 bytes 2026-02-19T20:22:40.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 285396992 bytes 2026-02-19T20:22:40.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 345869312 bytes 2026-02-19T20:22:40.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 391888896 bytes 2026-02-19T20:22:40.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 439150592 bytes 2026-02-19T20:22:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 466830336 bytes 2026-02-19T20:22:40.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 467431424 bytes 2026-02-19T20:22:40.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 519300096 bytes 2026-02-19T20:22:40.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 565222400 bytes 2026-02-19T20:22:40.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 626560000 bytes 2026-02-19T20:22:40.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 657880064 bytes 2026-02-19T20:22:40.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 719361024 bytes 2026-02-19T20:22:40.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 796141568 bytes 2026-02-19T20:22:40.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 834855936 bytes 2026-02-19T20:22:40.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 906157056 bytes 2026-02-19T20:22:40.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 982231040 bytes 2026-02-19T20:22:41.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1014452224 bytes 2026-02-19T20:22:41.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1067855872 bytes 2026-02-19T20:22:41.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1117259776 bytes 2026-02-19T20:22:41.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1117891584 bytes 2026-02-19T20:22:41.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1139884032 bytes 2026-02-19T20:22:41.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1190941696 bytes 2026-02-19T20:22:41.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1262938112 bytes 2026-02-19T20:22:41.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1318216704 bytes 2026-02-19T20:22:41.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1318840320 bytes 2026-02-19T20:22:41.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1375436800 bytes 2026-02-19T20:22:41.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1433410560 bytes 2026-02-19T20:22:41.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1532936192 bytes 2026-02-19T20:22:41.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1556955136 bytes 2026-02-19T20:22:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1597377536 bytes 2026-02-19T20:22:41.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1598005248 bytes 2026-02-19T20:22:41.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1609516032 bytes 2026-02-19T20:22:41.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1645012992 bytes 2026-02-19T20:22:41.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1687224320 bytes 2026-02-19T20:22:42.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1687833600 bytes 2026-02-19T20:22:42.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1731632128 bytes 2026-02-19T20:22:42.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1811650560 bytes 2026-02-19T20:22:42.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1873239040 bytes 2026-02-19T20:22:42.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1904686080 bytes 2026-02-19T20:22:42.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1929022464 bytes 2026-02-19T20:22:42.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1986521088 bytes 2026-02-19T20:22:42.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2082504704 bytes 2026-02-19T20:22:42.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2112932864 bytes 2026-02-19T20:22:42.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2187388928 bytes 2026-02-19T20:22:42.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2234750976 bytes 2026-02-19T20:22:42.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2298688512 bytes 2026-02-19T20:22:42.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2333947904 bytes 2026-02-19T20:22:42.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2405487616 bytes 2026-02-19T20:22:42.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2425623552 bytes 2026-02-19T20:22:42.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2463761408 bytes 2026-02-19T20:22:43.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2496940032 bytes 2026-02-19T20:22:43.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2523400192 bytes 2026-02-19T20:22:43.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2561935360 bytes 2026-02-19T20:22:43.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2602825728 bytes 2026-02-19T20:22:43.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2692929536 bytes 2026-02-19T20:22:43.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2728024064 bytes 2026-02-19T20:22:43.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2829886464 bytes 2026-02-19T20:22:43.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2881770496 bytes 2026-02-19T20:22:43.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2976451584 bytes 2026-02-19T20:22:43.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2991977472 bytes 2026-02-19T20:22:43.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3011356672 bytes 2026-02-19T20:22:43.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3053088768 bytes 2026-02-19T20:22:43.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3090692096 bytes 2026-02-19T20:22:44.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3160820736 bytes 2026-02-19T20:22:44.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3207743488 bytes 2026-02-19T20:22:44.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3287053312 bytes 2026-02-19T20:22:44.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3293319168 bytes 2026-02-19T20:22:44.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3312978944 bytes 2026-02-19T20:22:44.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3362768896 bytes 2026-02-19T20:22:44.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3389314048 bytes 2026-02-19T20:22:44.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3400900608 bytes 2026-02-19T20:22:44.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3480116224 bytes 2026-02-19T20:22:44.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3537349632 bytes 2026-02-19T20:22:44.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3553649664 bytes 2026-02-19T20:22:44.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3576621056 bytes 2026-02-19T20:22:44.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3625268224 bytes 2026-02-19T20:22:44.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3650432000 bytes 2026-02-19T20:22:44.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3669194752 bytes 2026-02-19T20:22:44.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3746330624 bytes 2026-02-19T20:22:44.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3775826944 bytes 2026-02-19T20:22:45.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3782223872 bytes 2026-02-19T20:22:45.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3803318272 bytes 2026-02-19T20:22:45.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3827621888 bytes 2026-02-19T20:22:45.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3893301248 bytes 2026-02-19T20:22:45.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3952472064 bytes 2026-02-19T20:22:45.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4028434432 bytes 2026-02-19T20:22:45.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4076154880 bytes 2026-02-19T20:22:45.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4144464896 bytes 2026-02-19T20:22:45.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4184044544 bytes 2026-02-19T20:22:45.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4212352000 bytes 2026-02-19T20:22:45.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4248592384 bytes 2026-02-19T20:22:45.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4271555584 bytes 2026-02-19T20:22:45.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4332004352 bytes 2026-02-19T20:22:45.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4361523200 bytes 2026-02-19T20:22:45.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4405889024 bytes 2026-02-19T20:22:45.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4468334592 bytes 2026-02-19T20:22:45.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4489986048 bytes 2026-02-19T20:22:46.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4518998016 bytes 2026-02-19T20:22:46.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4555275264 bytes 2026-02-19T20:22:46.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4567412736 bytes 2026-02-19T20:22:46.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4642562048 bytes 2026-02-19T20:22:46.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4666477568 bytes 2026-02-19T20:22:46.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4698986496 bytes 2026-02-19T20:22:46.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4767868928 bytes 2026-02-19T20:22:46.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4768449536 bytes 2026-02-19T20:22:46.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4779426816 bytes 2026-02-19T20:22:46.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4782554112 bytes 2026-02-19T20:22:46.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4796398592 bytes 2026-02-19T20:22:46.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4807698432 bytes 2026-02-19T20:22:46.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4841180160 bytes 2026-02-19T20:22:46.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:22:46.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:22:46.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:22:46.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:46.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_25 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_25 snapshot 128M 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:22:46.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_25 128M 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:46.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_25 2026-02-19T20:22:46.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:22:46.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:46.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:22:46.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:46.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_25 snapshot 2026-02-19T20:22:46.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:46.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:46.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:22:46.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:46.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_25 snapshot 2026-02-19T20:22:47.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:22:47.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_25 2026-02-19T20:22:47.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_25': 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1391a57ea338 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1391a57ea338 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:22:47.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:46 2026 2026-02-19T20:22:47.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:46 2026 2026-02-19T20:22:47.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:46 2026 2026-02-19T20:22:47.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:22:47.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:22:47.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: de88305c-0e9b-4c57-856e-bc3028b13752 2026-02-19T20:22:47.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:22:47.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:22:47.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_25 2026-02-19T20:22:47.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:47.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:47.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:22:47.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:22:47.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=11 2026-02-19T20:22:47.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:22:47.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_25 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:22:47.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 51368960 bytes 2026-02-19T20:22:47.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 91353088 bytes 2026-02-19T20:22:47.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 144789504 bytes 2026-02-19T20:22:47.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 164433920 bytes 2026-02-19T20:22:47.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 187604992 bytes 2026-02-19T20:22:47.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 206407680 bytes 2026-02-19T20:22:47.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 207032320 bytes 2026-02-19T20:22:48.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 273002496 bytes 2026-02-19T20:22:48.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 306494464 bytes 2026-02-19T20:22:48.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 346050560 bytes 2026-02-19T20:22:48.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 407342080 bytes 2026-02-19T20:22:48.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 437623808 bytes 2026-02-19T20:22:48.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 470905856 bytes 2026-02-19T20:22:48.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 529980416 bytes 2026-02-19T20:22:48.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 624268288 bytes 2026-02-19T20:22:48.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 645068800 bytes 2026-02-19T20:22:48.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 652899328 bytes 2026-02-19T20:22:48.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 675315712 bytes 2026-02-19T20:22:48.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 707486720 bytes 2026-02-19T20:22:48.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 713993216 bytes 2026-02-19T20:22:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 732936192 bytes 2026-02-19T20:22:48.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 780758016 bytes 2026-02-19T20:22:48.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 838294528 bytes 2026-02-19T20:22:48.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 860358656 bytes 2026-02-19T20:22:49.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 939014144 bytes 2026-02-19T20:22:49.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 964459520 bytes 2026-02-19T20:22:49.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 979842048 bytes 2026-02-19T20:22:49.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1066479616 bytes 2026-02-19T20:22:49.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1105981440 bytes 2026-02-19T20:22:49.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1144880128 bytes 2026-02-19T20:22:49.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1161961472 bytes 2026-02-19T20:22:49.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1177723904 bytes 2026-02-19T20:22:49.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1198472192 bytes 2026-02-19T20:22:49.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1206297600 bytes 2026-02-19T20:22:49.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1307962368 bytes 2026-02-19T20:22:49.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1380371456 bytes 2026-02-19T20:22:49.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1441990656 bytes 2026-02-19T20:22:49.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1463820288 bytes 2026-02-19T20:22:49.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1465594880 bytes 2026-02-19T20:22:49.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1525826560 bytes 2026-02-19T20:22:49.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1613903872 bytes 2026-02-19T20:22:49.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1623333888 bytes 2026-02-19T20:22:50.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1624100864 bytes 2026-02-19T20:22:50.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1662181376 bytes 2026-02-19T20:22:50.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1704553472 bytes 2026-02-19T20:22:50.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1751075840 bytes 2026-02-19T20:22:50.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1751672832 bytes 2026-02-19T20:22:50.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1809715200 bytes 2026-02-19T20:22:50.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1833164800 bytes 2026-02-19T20:22:50.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1879498752 bytes 2026-02-19T20:22:50.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1915362304 bytes 2026-02-19T20:22:50.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 1973335040 bytes 2026-02-19T20:22:50.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2028826624 bytes 2026-02-19T20:22:50.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2067259392 bytes 2026-02-19T20:22:50.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2136896512 bytes 2026-02-19T20:22:50.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2193691648 bytes 2026-02-19T20:22:50.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2278965248 bytes 2026-02-19T20:22:50.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2345529344 bytes 2026-02-19T20:22:51.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2372071424 bytes 2026-02-19T20:22:51.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2416288768 bytes 2026-02-19T20:22:51.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2453299200 bytes 2026-02-19T20:22:51.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2496611328 bytes 2026-02-19T20:22:51.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2517823488 bytes 2026-02-19T20:22:51.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2563163136 bytes 2026-02-19T20:22:51.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2587345920 bytes 2026-02-19T20:22:51.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2617397248 bytes 2026-02-19T20:22:51.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2628581376 bytes 2026-02-19T20:22:51.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2700600320 bytes 2026-02-19T20:22:51.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2759496704 bytes 2026-02-19T20:22:51.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2824491008 bytes 2026-02-19T20:22:51.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2828221440 bytes 2026-02-19T20:22:51.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2862026752 bytes 2026-02-19T20:22:51.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2895233024 bytes 2026-02-19T20:22:51.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2920584192 bytes 2026-02-19T20:22:51.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2962671616 bytes 2026-02-19T20:22:51.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2998340608 bytes 2026-02-19T20:22:52.049 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3053837312 bytes 2026-02-19T20:22:52.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3060926464 bytes 2026-02-19T20:22:52.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3116758016 bytes 2026-02-19T20:22:52.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3118343168 bytes 2026-02-19T20:22:52.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3147550720 bytes 2026-02-19T20:22:52.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3206423552 bytes 2026-02-19T20:22:52.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3275189248 bytes 2026-02-19T20:22:52.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3305205760 bytes 2026-02-19T20:22:52.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3403893760 bytes 2026-02-19T20:22:52.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3464045568 bytes 2026-02-19T20:22:52.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3546150912 bytes 2026-02-19T20:22:52.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3591603200 bytes 2026-02-19T20:22:52.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3621181440 bytes 2026-02-19T20:22:52.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3673850880 bytes 2026-02-19T20:22:52.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3712144384 bytes 2026-02-19T20:22:52.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3756909568 bytes 2026-02-19T20:22:53.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3810052096 bytes 2026-02-19T20:22:53.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3834874880 bytes 2026-02-19T20:22:53.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3866557440 bytes 2026-02-19T20:22:53.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3891771392 bytes 2026-02-19T20:22:53.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3892395008 bytes 2026-02-19T20:22:53.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3896250368 bytes 2026-02-19T20:22:53.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3974840320 bytes 2026-02-19T20:22:53.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4016886784 bytes 2026-02-19T20:22:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4058585088 bytes 2026-02-19T20:22:53.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4091329536 bytes 2026-02-19T20:22:53.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4160055296 bytes 2026-02-19T20:22:53.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4209101824 bytes 2026-02-19T20:22:53.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4252929024 bytes 2026-02-19T20:22:53.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4274501632 bytes 2026-02-19T20:22:53.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4292464640 bytes 2026-02-19T20:22:53.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4362913792 bytes 2026-02-19T20:22:53.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4363518976 bytes 2026-02-19T20:22:54.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4448529408 bytes 2026-02-19T20:22:54.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4496905216 bytes 2026-02-19T20:22:54.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4520962048 bytes 2026-02-19T20:22:54.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4560520192 bytes 2026-02-19T20:22:54.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4586129408 bytes 2026-02-19T20:22:54.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4623179776 bytes 2026-02-19T20:22:54.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4653844480 bytes 2026-02-19T20:22:54.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4729899008 bytes 2026-02-19T20:22:54.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4774819840 bytes 2026-02-19T20:22:54.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4787388416 bytes 2026-02-19T20:22:54.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4863899648 bytes 2026-02-19T20:22:54.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4896814080 bytes 2026-02-19T20:22:54.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4902555648 bytes 2026-02-19T20:22:54.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4912750592 bytes 2026-02-19T20:22:54.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4992601088 bytes 2026-02-19T20:22:54.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5022734336 bytes 2026-02-19T20:22:54.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5058776064 bytes 2026-02-19T20:22:54.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5084365824 bytes 2026-02-19T20:22:55.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5155872768 bytes 2026-02-19T20:22:55.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5197423616 bytes 2026-02-19T20:22:55.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5232280576 bytes 2026-02-19T20:22:55.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5250008064 bytes 2026-02-19T20:22:55.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5272391680 bytes 2026-02-19T20:22:55.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5307369472 bytes 2026-02-19T20:22:55.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5338281984 bytes 2026-02-19T20:22:55.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5373968384 bytes 2026-02-19T20:22:55.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5407675392 bytes 2026-02-19T20:22:55.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5440508928 bytes 2026-02-19T20:22:55.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5540194304 bytes 2026-02-19T20:22:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5590973440 bytes 2026-02-19T20:22:55.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5618469888 bytes 2026-02-19T20:22:56.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5734032384 bytes 2026-02-19T20:22:56.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5784673280 bytes 2026-02-19T20:22:56.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5839248384 bytes 2026-02-19T20:22:56.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5884370944 bytes 2026-02-19T20:22:56.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5925328896 bytes 2026-02-19T20:22:56.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5936857088 bytes 2026-02-19T20:22:56.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6047294464 bytes 2026-02-19T20:22:56.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6054545408 bytes 2026-02-19T20:22:56.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6064362496 bytes 2026-02-19T20:22:56.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6130104320 bytes 2026-02-19T20:22:56.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6188499968 bytes 2026-02-19T20:22:56.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6194607104 bytes 2026-02-19T20:22:56.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6224946176 bytes 2026-02-19T20:22:56.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6225550336 bytes 2026-02-19T20:22:56.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6235144192 bytes 2026-02-19T20:22:56.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6309907456 bytes 2026-02-19T20:22:56.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6339744768 bytes 2026-02-19T20:22:57.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6403434496 bytes 2026-02-19T20:22:57.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6431019008 bytes 2026-02-19T20:22:57.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6481844224 bytes 2026-02-19T20:22:57.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6525010944 bytes 2026-02-19T20:22:57.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6537749504 bytes 2026-02-19T20:22:57.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6577947648 bytes 2026-02-19T20:22:57.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6628358144 bytes 2026-02-19T20:22:57.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6672781312 bytes 2026-02-19T20:22:57.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6711316480 bytes 2026-02-19T20:22:57.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6772697088 bytes 2026-02-19T20:22:57.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6810983424 bytes 2026-02-19T20:22:57.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6852387840 bytes 2026-02-19T20:22:57.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6920406016 bytes 2026-02-19T20:22:57.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6929537024 bytes 2026-02-19T20:22:57.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7005377536 bytes 2026-02-19T20:22:57.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7100240896 bytes 2026-02-19T20:22:58.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7167226880 bytes 2026-02-19T20:22:58.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7241470976 bytes 2026-02-19T20:22:58.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7280745472 bytes 2026-02-19T20:22:58.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7326415872 bytes 2026-02-19T20:22:58.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7412641792 bytes 2026-02-19T20:22:58.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7443421184 bytes 2026-02-19T20:22:58.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7515210752 bytes 2026-02-19T20:22:58.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7517843456 bytes 2026-02-19T20:22:58.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7550328832 bytes 2026-02-19T20:22:58.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:22:58.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:22:58.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:22:58.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:58.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:22:58.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_26 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_26 snapshot 128M 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:22:58.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_26 128M 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:22:58.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:22:58.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:22:58.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_26 2026-02-19T20:22:58.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:22:58.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:58.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:22:58.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:22:58.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_26 snapshot 2026-02-19T20:22:58.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:58.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:58.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:22:58.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:22:58.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_26 snapshot 2026-02-19T20:22:59.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:22:59.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_26 2026-02-19T20:22:59.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_26': 2026-02-19T20:22:59.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:22:59.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:22:59.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 139d93d225fb 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.139d93d225fb 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:58 2026 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:58 2026 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:58 2026 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:22:59.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:22:59.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 807691bb-e01b-4205-9b22-0a3c88b5aa24 2026-02-19T20:22:59.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:22:59.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:22:59.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_26 2026-02-19T20:22:59.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:22:59.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:22:59.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:22:59.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:22:59.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=5 2026-02-19T20:22:59.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:22:59.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_26 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:22:59.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 30977024 bytes 2026-02-19T20:22:59.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 88886272 bytes 2026-02-19T20:22:59.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 114332672 bytes 2026-02-19T20:22:59.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 185863168 bytes 2026-02-19T20:22:59.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 187852800 bytes 2026-02-19T20:22:59.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 255794176 bytes 2026-02-19T20:23:00.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 319792128 bytes 2026-02-19T20:23:00.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 368164864 bytes 2026-02-19T20:23:00.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 391990272 bytes 2026-02-19T20:23:00.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 433211392 bytes 2026-02-19T20:23:00.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 464578560 bytes 2026-02-19T20:23:00.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 487639040 bytes 2026-02-19T20:23:00.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 541492224 bytes 2026-02-19T20:23:00.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 605316096 bytes 2026-02-19T20:23:00.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 633847808 bytes 2026-02-19T20:23:00.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 694623232 bytes 2026-02-19T20:23:00.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 826014720 bytes 2026-02-19T20:23:00.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 848382976 bytes 2026-02-19T20:23:00.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 848984064 bytes 2026-02-19T20:23:00.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 914189312 bytes 2026-02-19T20:23:00.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 950936576 bytes 2026-02-19T20:23:01.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1005895680 bytes 2026-02-19T20:23:01.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1006516224 bytes 2026-02-19T20:23:01.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1034240000 bytes 2026-02-19T20:23:01.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1105421312 bytes 2026-02-19T20:23:01.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1132387328 bytes 2026-02-19T20:23:01.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1136285696 bytes 2026-02-19T20:23:01.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1137724416 bytes 2026-02-19T20:23:01.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1173624832 bytes 2026-02-19T20:23:01.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1230832640 bytes 2026-02-19T20:23:01.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1270507520 bytes 2026-02-19T20:23:01.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1314020352 bytes 2026-02-19T20:23:01.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1327773696 bytes 2026-02-19T20:23:01.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1367005184 bytes 2026-02-19T20:23:01.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1395110912 bytes 2026-02-19T20:23:01.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1410430976 bytes 2026-02-19T20:23:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1442354176 bytes 2026-02-19T20:23:01.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1505241088 bytes 2026-02-19T20:23:01.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1566040064 bytes 2026-02-19T20:23:02.049 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1609214976 bytes 2026-02-19T20:23:02.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1616507904 bytes 2026-02-19T20:23:02.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1673328640 bytes 2026-02-19T20:23:02.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1675220992 bytes 2026-02-19T20:23:02.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1747518464 bytes 2026-02-19T20:23:02.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1844974592 bytes 2026-02-19T20:23:02.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1874558976 bytes 2026-02-19T20:23:02.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1895138304 bytes 2026-02-19T20:23:02.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1908283392 bytes 2026-02-19T20:23:02.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1956407296 bytes 2026-02-19T20:23:02.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1998304256 bytes 2026-02-19T20:23:02.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2034182144 bytes 2026-02-19T20:23:02.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2079405056 bytes 2026-02-19T20:23:02.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2081615872 bytes 2026-02-19T20:23:02.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2090255360 bytes 2026-02-19T20:23:02.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2106171392 bytes 2026-02-19T20:23:02.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2127097856 bytes 2026-02-19T20:23:02.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2184478720 bytes 2026-02-19T20:23:03.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2241024000 bytes 2026-02-19T20:23:03.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2280682496 bytes 2026-02-19T20:23:03.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2369191936 bytes 2026-02-19T20:23:03.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2417547264 bytes 2026-02-19T20:23:03.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2460288000 bytes 2026-02-19T20:23:03.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2518858752 bytes 2026-02-19T20:23:03.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2550592512 bytes 2026-02-19T20:23:03.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2586298368 bytes 2026-02-19T20:23:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2629337088 bytes 2026-02-19T20:23:03.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2696724480 bytes 2026-02-19T20:23:03.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2777224192 bytes 2026-02-19T20:23:03.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2800642048 bytes 2026-02-19T20:23:03.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2809366528 bytes 2026-02-19T20:23:03.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2830430208 bytes 2026-02-19T20:23:03.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2855849984 bytes 2026-02-19T20:23:04.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2907005952 bytes 2026-02-19T20:23:04.083 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2920513536 bytes 2026-02-19T20:23:04.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2978321408 bytes 2026-02-19T20:23:04.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2982443008 bytes 2026-02-19T20:23:04.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3085394944 bytes 2026-02-19T20:23:04.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3137424384 bytes 2026-02-19T20:23:04.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3165096960 bytes 2026-02-19T20:23:04.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3177028608 bytes 2026-02-19T20:23:04.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3192436736 bytes 2026-02-19T20:23:04.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3199280128 bytes 2026-02-19T20:23:04.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3255714816 bytes 2026-02-19T20:23:04.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:23:04.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_27 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_27 snapshot 128M 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_27 128M 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:23:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:23:04.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:04.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_27 2026-02-19T20:23:04.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:23:04.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:04.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:23:04.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:04.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_27 snapshot 2026-02-19T20:23:04.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:04.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:04.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:23:04.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:04.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_27 snapshot 2026-02-19T20:23:05.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:23:05.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_27 2026-02-19T20:23:05.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_27': 2026-02-19T20:23:05.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:23:05.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:23:05.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:23:05.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 13a9984b986e 2026-02-19T20:23:05.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.13a9984b986e 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:04 2026 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:04 2026 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:04 2026 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 26aa151c-5c6e-407f-84a4-06e391048d6e 2026-02-19T20:23:05.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:23:05.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:23:05.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_27 2026-02-19T20:23:05.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:05.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:05.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:23:05.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:23:05.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:23:05.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:23:05.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_27 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:23:05.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 49695744 bytes 2026-02-19T20:23:05.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 114862080 bytes 2026-02-19T20:23:05.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 116800512 bytes 2026-02-19T20:23:05.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 210442240 bytes 2026-02-19T20:23:05.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 299397120 bytes 2026-02-19T20:23:06.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 359571456 bytes 2026-02-19T20:23:06.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 432955392 bytes 2026-02-19T20:23:06.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 458364928 bytes 2026-02-19T20:23:06.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 478390272 bytes 2026-02-19T20:23:06.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 479015936 bytes 2026-02-19T20:23:06.292 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 487503872 bytes 2026-02-19T20:23:06.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 514982912 bytes 2026-02-19T20:23:06.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 594978816 bytes 2026-02-19T20:23:06.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 652070912 bytes 2026-02-19T20:23:06.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 731636736 bytes 2026-02-19T20:23:06.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 772848640 bytes 2026-02-19T20:23:06.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 778673152 bytes 2026-02-19T20:23:06.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 834782208 bytes 2026-02-19T20:23:06.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 886693888 bytes 2026-02-19T20:23:06.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 958057472 bytes 2026-02-19T20:23:06.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 958642176 bytes 2026-02-19T20:23:07.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1005303808 bytes 2026-02-19T20:23:07.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1026164736 bytes 2026-02-19T20:23:07.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1062740992 bytes 2026-02-19T20:23:07.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1101721600 bytes 2026-02-19T20:23:07.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1170839552 bytes 2026-02-19T20:23:07.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1193277440 bytes 2026-02-19T20:23:07.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1228911616 bytes 2026-02-19T20:23:07.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1287563264 bytes 2026-02-19T20:23:07.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1381783552 bytes 2026-02-19T20:23:07.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1421905920 bytes 2026-02-19T20:23:07.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1486456832 bytes 2026-02-19T20:23:07.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1540562944 bytes 2026-02-19T20:23:07.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1572829184 bytes 2026-02-19T20:23:07.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1601065984 bytes 2026-02-19T20:23:07.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1625835520 bytes 2026-02-19T20:23:07.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1706704896 bytes 2026-02-19T20:23:08.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1707321344 bytes 2026-02-19T20:23:08.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1771300864 bytes 2026-02-19T20:23:08.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1776088064 bytes 2026-02-19T20:23:08.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1805319168 bytes 2026-02-19T20:23:08.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1863687168 bytes 2026-02-19T20:23:08.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1910188032 bytes 2026-02-19T20:23:08.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1997465600 bytes 2026-02-19T20:23:08.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2049323008 bytes 2026-02-19T20:23:08.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2077614080 bytes 2026-02-19T20:23:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2133513216 bytes 2026-02-19T20:23:08.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2151910400 bytes 2026-02-19T20:23:08.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2187020288 bytes 2026-02-19T20:23:08.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2242112512 bytes 2026-02-19T20:23:08.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2279578624 bytes 2026-02-19T20:23:08.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2320833536 bytes 2026-02-19T20:23:08.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2335885312 bytes 2026-02-19T20:23:08.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2376866816 bytes 2026-02-19T20:23:09.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2412149760 bytes 2026-02-19T20:23:09.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2434066432 bytes 2026-02-19T20:23:09.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2505233408 bytes 2026-02-19T20:23:09.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2511299584 bytes 2026-02-19T20:23:09.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2544772096 bytes 2026-02-19T20:23:09.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2578230272 bytes 2026-02-19T20:23:09.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2699786240 bytes 2026-02-19T20:23:09.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2774396928 bytes 2026-02-19T20:23:09.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2796942336 bytes 2026-02-19T20:23:09.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2839327744 bytes 2026-02-19T20:23:09.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2866951168 bytes 2026-02-19T20:23:09.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2944202752 bytes 2026-02-19T20:23:09.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2951111680 bytes 2026-02-19T20:23:09.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3006743552 bytes 2026-02-19T20:23:09.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3037661184 bytes 2026-02-19T20:23:09.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3048682496 bytes 2026-02-19T20:23:09.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3072080896 bytes 2026-02-19T20:23:10.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3095979008 bytes 2026-02-19T20:23:10.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3184291840 bytes 2026-02-19T20:23:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3223870464 bytes 2026-02-19T20:23:10.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3244454912 bytes 2026-02-19T20:23:10.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3302963200 bytes 2026-02-19T20:23:10.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3356558336 bytes 2026-02-19T20:23:10.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3412530176 bytes 2026-02-19T20:23:10.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3413124096 bytes 2026-02-19T20:23:10.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3446796288 bytes 2026-02-19T20:23:10.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3460563968 bytes 2026-02-19T20:23:10.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3461184512 bytes 2026-02-19T20:23:10.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3508644864 bytes 2026-02-19T20:23:10.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3542054912 bytes 2026-02-19T20:23:10.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3547160576 bytes 2026-02-19T20:23:10.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3583016960 bytes 2026-02-19T20:23:10.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3591948288 bytes 2026-02-19T20:23:10.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3647223808 bytes 2026-02-19T20:23:10.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3673628672 bytes 2026-02-19T20:23:11.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3714623488 bytes 2026-02-19T20:23:11.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3757100032 bytes 2026-02-19T20:23:11.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3817829376 bytes 2026-02-19T20:23:11.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3911177216 bytes 2026-02-19T20:23:11.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3946278912 bytes 2026-02-19T20:23:11.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3960693760 bytes 2026-02-19T20:23:11.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4030643200 bytes 2026-02-19T20:23:11.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4089655296 bytes 2026-02-19T20:23:11.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4115829760 bytes 2026-02-19T20:23:11.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4182945792 bytes 2026-02-19T20:23:11.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4208477184 bytes 2026-02-19T20:23:11.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4237142016 bytes 2026-02-19T20:23:11.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4333272064 bytes 2026-02-19T20:23:11.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4387018752 bytes 2026-02-19T20:23:11.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4416794624 bytes 2026-02-19T20:23:12.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4417388544 bytes 2026-02-19T20:23:12.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4466906112 bytes 2026-02-19T20:23:12.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4547783680 bytes 2026-02-19T20:23:12.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4548403200 bytes 2026-02-19T20:23:12.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4607505408 bytes 2026-02-19T20:23:12.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4656465920 bytes 2026-02-19T20:23:12.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4682514432 bytes 2026-02-19T20:23:12.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4711816192 bytes 2026-02-19T20:23:12.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4723187712 bytes 2026-02-19T20:23:12.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4763691008 bytes 2026-02-19T20:23:12.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4798127104 bytes 2026-02-19T20:23:12.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4837066752 bytes 2026-02-19T20:23:12.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4866387968 bytes 2026-02-19T20:23:12.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4904981504 bytes 2026-02-19T20:23:12.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4907978752 bytes 2026-02-19T20:23:12.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4942059520 bytes 2026-02-19T20:23:12.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4988603392 bytes 2026-02-19T20:23:12.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5033339904 bytes 2026-02-19T20:23:13.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5069363200 bytes 2026-02-19T20:23:13.077 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5092172800 bytes 2026-02-19T20:23:13.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5147471872 bytes 2026-02-19T20:23:13.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5188247552 bytes 2026-02-19T20:23:13.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5218683904 bytes 2026-02-19T20:23:13.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5255048192 bytes 2026-02-19T20:23:13.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5274387456 bytes 2026-02-19T20:23:13.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5303489536 bytes 2026-02-19T20:23:13.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5342198784 bytes 2026-02-19T20:23:13.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5361950720 bytes 2026-02-19T20:23:13.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5392045056 bytes 2026-02-19T20:23:13.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5444517888 bytes 2026-02-19T20:23:13.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5535525888 bytes 2026-02-19T20:23:13.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5560481792 bytes 2026-02-19T20:23:14.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5591142400 bytes 2026-02-19T20:23:14.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5650705408 bytes 2026-02-19T20:23:14.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5726028800 bytes 2026-02-19T20:23:14.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5759043584 bytes 2026-02-19T20:23:14.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5796403200 bytes 2026-02-19T20:23:14.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5819566080 bytes 2026-02-19T20:23:14.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5907201024 bytes 2026-02-19T20:23:14.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5947774976 bytes 2026-02-19T20:23:14.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5953608704 bytes 2026-02-19T20:23:14.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5994088448 bytes 2026-02-19T20:23:14.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6040439808 bytes 2026-02-19T20:23:14.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6115870720 bytes 2026-02-19T20:23:14.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6169849856 bytes 2026-02-19T20:23:14.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6205178880 bytes 2026-02-19T20:23:14.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6226336768 bytes 2026-02-19T20:23:14.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6226947072 bytes 2026-02-19T20:23:14.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6232697856 bytes 2026-02-19T20:23:15.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6282002432 bytes 2026-02-19T20:23:15.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6324768768 bytes 2026-02-19T20:23:15.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6337957888 bytes 2026-02-19T20:23:15.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6376132608 bytes 2026-02-19T20:23:15.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6456554496 bytes 2026-02-19T20:23:15.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6463756288 bytes 2026-02-19T20:23:15.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6501235712 bytes 2026-02-19T20:23:15.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6590171136 bytes 2026-02-19T20:23:15.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6606957568 bytes 2026-02-19T20:23:15.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6636029952 bytes 2026-02-19T20:23:15.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6734067712 bytes 2026-02-19T20:23:15.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6787954688 bytes 2026-02-19T20:23:15.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6848537600 bytes 2026-02-19T20:23:15.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6894671872 bytes 2026-02-19T20:23:15.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6903524352 bytes 2026-02-19T20:23:15.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6957714432 bytes 2026-02-19T20:23:16.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6974073856 bytes 2026-02-19T20:23:16.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7067017216 bytes 2026-02-19T20:23:16.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7074992128 bytes 2026-02-19T20:23:16.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7119568896 bytes 2026-02-19T20:23:16.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7157126144 bytes 2026-02-19T20:23:16.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7209367552 bytes 2026-02-19T20:23:16.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7242667008 bytes 2026-02-19T20:23:16.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7278373888 bytes 2026-02-19T20:23:16.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7323574272 bytes 2026-02-19T20:23:16.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7421134848 bytes 2026-02-19T20:23:16.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7545253888 bytes 2026-02-19T20:23:16.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7594181632 bytes 2026-02-19T20:23:16.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7649600512 bytes 2026-02-19T20:23:16.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7662539776 bytes 2026-02-19T20:23:16.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7744799744 bytes 2026-02-19T20:23:17.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7774145536 bytes 2026-02-19T20:23:17.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7785611264 bytes 2026-02-19T20:23:17.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7842178048 bytes 2026-02-19T20:23:17.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7891173376 bytes 2026-02-19T20:23:17.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7907940352 bytes 2026-02-19T20:23:17.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7940589568 bytes 2026-02-19T20:23:17.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7961978880 bytes 2026-02-19T20:23:17.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7965670400 bytes 2026-02-19T20:23:17.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8022617088 bytes 2026-02-19T20:23:17.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8047101952 bytes 2026-02-19T20:23:17.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8085452800 bytes 2026-02-19T20:23:17.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8088089600 bytes 2026-02-19T20:23:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:23:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:23:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:23:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:23:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_28 2026-02-19T20:23:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_28 snapshot 128M 2026-02-19T20:23:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:17.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_28 128M 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:17.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:17.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:23:17.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:17.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:23:17.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:23:17.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:23:17.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:17.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_28 2026-02-19T20:23:17.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:23:17.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:17.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:23:17.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:17.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_28 snapshot 2026-02-19T20:23:17.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:17.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:17.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:23:17.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:17.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_28 snapshot 2026-02-19T20:23:18.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:23:18.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_28 2026-02-19T20:23:18.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_28': 2026-02-19T20:23:18.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:23:18.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:23:18.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:23:18.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 13b5328aef57 2026-02-19T20:23:18.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.13b5328aef57 2026-02-19T20:23:18.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:17 2026 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:17 2026 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:17 2026 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 0df66c2e-adfc-4e01-b702-4d0a8bd1c719 2026-02-19T20:23:18.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:23:18.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:23:18.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_28 2026-02-19T20:23:18.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:18.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:18.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:23:18.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:23:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=11 2026-02-19T20:23:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:23:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_28 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:23:18.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 76567552 bytes 2026-02-19T20:23:18.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 146332672 bytes 2026-02-19T20:23:18.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 175318016 bytes 2026-02-19T20:23:18.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 210974720 bytes 2026-02-19T20:23:18.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 229430272 bytes 2026-02-19T20:23:19.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 230028288 bytes 2026-02-19T20:23:19.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 236972032 bytes 2026-02-19T20:23:19.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 262036480 bytes 2026-02-19T20:23:19.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 262662144 bytes 2026-02-19T20:23:19.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 312884224 bytes 2026-02-19T20:23:19.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 323257344 bytes 2026-02-19T20:23:19.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 388057088 bytes 2026-02-19T20:23:19.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 459773952 bytes 2026-02-19T20:23:19.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 470732800 bytes 2026-02-19T20:23:19.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 508243968 bytes 2026-02-19T20:23:19.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 595400704 bytes 2026-02-19T20:23:19.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 608343040 bytes 2026-02-19T20:23:19.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 625804288 bytes 2026-02-19T20:23:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 678852608 bytes 2026-02-19T20:23:19.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 696712192 bytes 2026-02-19T20:23:19.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 711188480 bytes 2026-02-19T20:23:19.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 755122176 bytes 2026-02-19T20:23:19.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 770200576 bytes 2026-02-19T20:23:20.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 817796096 bytes 2026-02-19T20:23:20.077 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 842302464 bytes 2026-02-19T20:23:20.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 913521664 bytes 2026-02-19T20:23:20.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 964913152 bytes 2026-02-19T20:23:20.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1012714496 bytes 2026-02-19T20:23:20.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1039965184 bytes 2026-02-19T20:23:20.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1068209152 bytes 2026-02-19T20:23:20.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1131177984 bytes 2026-02-19T20:23:20.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1170135040 bytes 2026-02-19T20:23:20.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1212146688 bytes 2026-02-19T20:23:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1256803328 bytes 2026-02-19T20:23:20.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1303517184 bytes 2026-02-19T20:23:20.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1327687680 bytes 2026-02-19T20:23:20.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1344971776 bytes 2026-02-19T20:23:20.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1359430656 bytes 2026-02-19T20:23:20.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1418888192 bytes 2026-02-19T20:23:21.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1443657728 bytes 2026-02-19T20:23:21.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1475511296 bytes 2026-02-19T20:23:21.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1506230272 bytes 2026-02-19T20:23:21.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1513152512 bytes 2026-02-19T20:23:21.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1577476096 bytes 2026-02-19T20:23:21.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1578087424 bytes 2026-02-19T20:23:21.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1601348608 bytes 2026-02-19T20:23:21.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1639773184 bytes 2026-02-19T20:23:21.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1715817472 bytes 2026-02-19T20:23:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1735312384 bytes 2026-02-19T20:23:21.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1840418816 bytes 2026-02-19T20:23:21.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1887631360 bytes 2026-02-19T20:23:21.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 1957797888 bytes 2026-02-19T20:23:21.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 1980712960 bytes 2026-02-19T20:23:21.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 1999056896 bytes 2026-02-19T20:23:21.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2025808896 bytes 2026-02-19T20:23:21.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2068576256 bytes 2026-02-19T20:23:22.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2079834112 bytes 2026-02-19T20:23:22.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2106564608 bytes 2026-02-19T20:23:22.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2152582144 bytes 2026-02-19T20:23:22.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2233454592 bytes 2026-02-19T20:23:22.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2283386880 bytes 2026-02-19T20:23:22.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2365338624 bytes 2026-02-19T20:23:22.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2410530816 bytes 2026-02-19T20:23:22.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2472336384 bytes 2026-02-19T20:23:22.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2530507776 bytes 2026-02-19T20:23:22.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2561634304 bytes 2026-02-19T20:23:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2575058944 bytes 2026-02-19T20:23:22.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2638073856 bytes 2026-02-19T20:23:22.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2649040896 bytes 2026-02-19T20:23:22.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2700193792 bytes 2026-02-19T20:23:22.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2755910656 bytes 2026-02-19T20:23:23.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2803742720 bytes 2026-02-19T20:23:23.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2817433600 bytes 2026-02-19T20:23:23.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2911794176 bytes 2026-02-19T20:23:23.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2987427840 bytes 2026-02-19T20:23:23.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3035381760 bytes 2026-02-19T20:23:23.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3082634240 bytes 2026-02-19T20:23:23.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3092058112 bytes 2026-02-19T20:23:23.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3092670464 bytes 2026-02-19T20:23:23.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3116715008 bytes 2026-02-19T20:23:23.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3138510848 bytes 2026-02-19T20:23:23.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3164015616 bytes 2026-02-19T20:23:23.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3218833408 bytes 2026-02-19T20:23:23.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3278601216 bytes 2026-02-19T20:23:23.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3349090304 bytes 2026-02-19T20:23:23.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3389393920 bytes 2026-02-19T20:23:23.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3411919872 bytes 2026-02-19T20:23:24.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3439800320 bytes 2026-02-19T20:23:24.083 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3462020096 bytes 2026-02-19T20:23:24.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3493210112 bytes 2026-02-19T20:23:24.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3541944320 bytes 2026-02-19T20:23:24.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3567434752 bytes 2026-02-19T20:23:24.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3588355072 bytes 2026-02-19T20:23:24.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3655514112 bytes 2026-02-19T20:23:24.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3685195776 bytes 2026-02-19T20:23:24.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3723392000 bytes 2026-02-19T20:23:24.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3740798976 bytes 2026-02-19T20:23:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3765187584 bytes 2026-02-19T20:23:24.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3846407168 bytes 2026-02-19T20:23:24.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 3882437632 bytes 2026-02-19T20:23:24.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 3919670272 bytes 2026-02-19T20:23:24.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 3950849024 bytes 2026-02-19T20:23:24.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 3978307584 bytes 2026-02-19T20:23:25.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4057668608 bytes 2026-02-19T20:23:25.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4064580608 bytes 2026-02-19T20:23:25.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4088120320 bytes 2026-02-19T20:23:25.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4175083520 bytes 2026-02-19T20:23:25.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4238201856 bytes 2026-02-19T20:23:25.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4319489024 bytes 2026-02-19T20:23:25.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4364341248 bytes 2026-02-19T20:23:25.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4413840384 bytes 2026-02-19T20:23:25.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4414484480 bytes 2026-02-19T20:23:25.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4442569728 bytes 2026-02-19T20:23:25.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4478363648 bytes 2026-02-19T20:23:25.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4527886336 bytes 2026-02-19T20:23:25.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4570941440 bytes 2026-02-19T20:23:25.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4654598144 bytes 2026-02-19T20:23:26.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4774004736 bytes 2026-02-19T20:23:26.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4858768384 bytes 2026-02-19T20:23:26.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4915981312 bytes 2026-02-19T20:23:26.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4970659840 bytes 2026-02-19T20:23:26.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4994831360 bytes 2026-02-19T20:23:26.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5040849920 bytes 2026-02-19T20:23:26.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5041461248 bytes 2026-02-19T20:23:26.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5056184320 bytes 2026-02-19T20:23:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5125080064 bytes 2026-02-19T20:23:26.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5170969600 bytes 2026-02-19T20:23:26.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5201298432 bytes 2026-02-19T20:23:26.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5251185664 bytes 2026-02-19T20:23:26.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5272739840 bytes 2026-02-19T20:23:26.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5300636672 bytes 2026-02-19T20:23:26.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5340175360 bytes 2026-02-19T20:23:27.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5413412864 bytes 2026-02-19T20:23:27.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5435933696 bytes 2026-02-19T20:23:27.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5445874688 bytes 2026-02-19T20:23:27.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5456026624 bytes 2026-02-19T20:23:27.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5515183104 bytes 2026-02-19T20:23:27.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5540016128 bytes 2026-02-19T20:23:27.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5546482688 bytes 2026-02-19T20:23:27.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5603453952 bytes 2026-02-19T20:23:27.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5633685504 bytes 2026-02-19T20:23:27.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5641840640 bytes 2026-02-19T20:23:27.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5681382400 bytes 2026-02-19T20:23:27.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5772725248 bytes 2026-02-19T20:23:27.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5786730496 bytes 2026-02-19T20:23:27.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5810920448 bytes 2026-02-19T20:23:27.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5892050944 bytes 2026-02-19T20:23:27.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5901513728 bytes 2026-02-19T20:23:27.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 5943250944 bytes 2026-02-19T20:23:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6033353728 bytes 2026-02-19T20:23:28.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6057379840 bytes 2026-02-19T20:23:28.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6074662912 bytes 2026-02-19T20:23:28.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6129369088 bytes 2026-02-19T20:23:28.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6152245248 bytes 2026-02-19T20:23:28.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6152848384 bytes 2026-02-19T20:23:28.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6195526656 bytes 2026-02-19T20:23:28.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6198572032 bytes 2026-02-19T20:23:28.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6245855232 bytes 2026-02-19T20:23:28.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6289831936 bytes 2026-02-19T20:23:28.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6349506560 bytes 2026-02-19T20:23:28.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6370620416 bytes 2026-02-19T20:23:28.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6472520704 bytes 2026-02-19T20:23:28.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6508920832 bytes 2026-02-19T20:23:28.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6534890496 bytes 2026-02-19T20:23:28.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6565612544 bytes 2026-02-19T20:23:28.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6591955968 bytes 2026-02-19T20:23:29.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6617285632 bytes 2026-02-19T20:23:29.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6652353536 bytes 2026-02-19T20:23:29.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6659856384 bytes 2026-02-19T20:23:29.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6688590848 bytes 2026-02-19T20:23:29.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6710918144 bytes 2026-02-19T20:23:29.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 6747113472 bytes 2026-02-19T20:23:29.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 6784099328 bytes 2026-02-19T20:23:29.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 6823263232 bytes 2026-02-19T20:23:29.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 6823867392 bytes 2026-02-19T20:23:29.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 6906294272 bytes 2026-02-19T20:23:29.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 6979024896 bytes 2026-02-19T20:23:29.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:23:29.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:23:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:23:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:23:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_29 2026-02-19T20:23:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_29 snapshot 128M 2026-02-19T20:23:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_29 128M 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:23:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:23:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:23:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:23:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_29 2026-02-19T20:23:29.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:23:29.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:29.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:23:29.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:29.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_29 snapshot 2026-02-19T20:23:29.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:29.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:29.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:23:29.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:29.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_29 snapshot 2026-02-19T20:23:30.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:23:30.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_29 2026-02-19T20:23:30.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_29': 2026-02-19T20:23:30.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 13c13bbd0644 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.13c13bbd0644 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:29 2026 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:29 2026 2026-02-19T20:23:30.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:29 2026 2026-02-19T20:23:30.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:23:30.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:23:30.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 2df60094-cd0e-43df-88af-ea85d6cc9a96 2026-02-19T20:23:30.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:23:30.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:23:30.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_29 2026-02-19T20:23:30.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:30.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:30.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:23:30.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:23:30.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=9 2026-02-19T20:23:30.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:23:30.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_29 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:23:30.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 25295872 bytes 2026-02-19T20:23:30.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 91475968 bytes 2026-02-19T20:23:30.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 93248512 bytes 2026-02-19T20:23:30.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 103033856 bytes 2026-02-19T20:23:30.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 205491200 bytes 2026-02-19T20:23:30.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 224078848 bytes 2026-02-19T20:23:31.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 274169856 bytes 2026-02-19T20:23:31.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 300932096 bytes 2026-02-19T20:23:31.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 340018176 bytes 2026-02-19T20:23:31.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 377327616 bytes 2026-02-19T20:23:31.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 393749504 bytes 2026-02-19T20:23:31.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 424109056 bytes 2026-02-19T20:23:31.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 460317696 bytes 2026-02-19T20:23:31.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 499723264 bytes 2026-02-19T20:23:31.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 539650048 bytes 2026-02-19T20:23:31.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 566426624 bytes 2026-02-19T20:23:31.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 598972416 bytes 2026-02-19T20:23:31.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 686696448 bytes 2026-02-19T20:23:31.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 687315968 bytes 2026-02-19T20:23:31.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 730494976 bytes 2026-02-19T20:23:31.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 757326848 bytes 2026-02-19T20:23:31.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 808249344 bytes 2026-02-19T20:23:31.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 817836032 bytes 2026-02-19T20:23:32.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 866235392 bytes 2026-02-19T20:23:32.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 932608000 bytes 2026-02-19T20:23:32.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1022837760 bytes 2026-02-19T20:23:32.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1086995456 bytes 2026-02-19T20:23:32.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1142810624 bytes 2026-02-19T20:23:32.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1186095104 bytes 2026-02-19T20:23:32.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1271376896 bytes 2026-02-19T20:23:32.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1316047872 bytes 2026-02-19T20:23:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1405123584 bytes 2026-02-19T20:23:32.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1475060736 bytes 2026-02-19T20:23:32.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1487944704 bytes 2026-02-19T20:23:32.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1496961024 bytes 2026-02-19T20:23:32.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1547804672 bytes 2026-02-19T20:23:32.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1577142272 bytes 2026-02-19T20:23:33.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1627204608 bytes 2026-02-19T20:23:33.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1677763584 bytes 2026-02-19T20:23:33.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1719342080 bytes 2026-02-19T20:23:33.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1787817984 bytes 2026-02-19T20:23:33.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1795355648 bytes 2026-02-19T20:23:33.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1887884288 bytes 2026-02-19T20:23:33.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1899927552 bytes 2026-02-19T20:23:33.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1937843200 bytes 2026-02-19T20:23:33.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1982252032 bytes 2026-02-19T20:23:33.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2017306624 bytes 2026-02-19T20:23:33.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2074093568 bytes 2026-02-19T20:23:33.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2086250496 bytes 2026-02-19T20:23:33.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2106221568 bytes 2026-02-19T20:23:33.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2129255424 bytes 2026-02-19T20:23:33.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2181946368 bytes 2026-02-19T20:23:33.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2264953856 bytes 2026-02-19T20:23:33.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2265573376 bytes 2026-02-19T20:23:34.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2316400640 bytes 2026-02-19T20:23:34.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2317038592 bytes 2026-02-19T20:23:34.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2368418816 bytes 2026-02-19T20:23:34.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2406179840 bytes 2026-02-19T20:23:34.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2438660096 bytes 2026-02-19T20:23:34.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2444238848 bytes 2026-02-19T20:23:34.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2485728256 bytes 2026-02-19T20:23:34.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2540379136 bytes 2026-02-19T20:23:34.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2589316096 bytes 2026-02-19T20:23:34.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2631224320 bytes 2026-02-19T20:23:34.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2671068160 bytes 2026-02-19T20:23:34.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2753897472 bytes 2026-02-19T20:23:34.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2822812672 bytes 2026-02-19T20:23:34.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2967927808 bytes 2026-02-19T20:23:34.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3026596864 bytes 2026-02-19T20:23:35.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3044498432 bytes 2026-02-19T20:23:35.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3054982144 bytes 2026-02-19T20:23:35.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3062685696 bytes 2026-02-19T20:23:35.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3097231360 bytes 2026-02-19T20:23:35.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3144482816 bytes 2026-02-19T20:23:35.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3146175488 bytes 2026-02-19T20:23:35.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3166596096 bytes 2026-02-19T20:23:35.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3173164032 bytes 2026-02-19T20:23:35.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3213361152 bytes 2026-02-19T20:23:35.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3256261632 bytes 2026-02-19T20:23:35.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3256885248 bytes 2026-02-19T20:23:35.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3314956288 bytes 2026-02-19T20:23:35.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3342808064 bytes 2026-02-19T20:23:35.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3369858048 bytes 2026-02-19T20:23:35.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3395170304 bytes 2026-02-19T20:23:35.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3401993216 bytes 2026-02-19T20:23:35.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3425773568 bytes 2026-02-19T20:23:35.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3435445248 bytes 2026-02-19T20:23:35.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3447252992 bytes 2026-02-19T20:23:35.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3522176000 bytes 2026-02-19T20:23:36.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3584088064 bytes 2026-02-19T20:23:36.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3628466176 bytes 2026-02-19T20:23:36.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3708370944 bytes 2026-02-19T20:23:36.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3772688384 bytes 2026-02-19T20:23:36.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3843608576 bytes 2026-02-19T20:23:36.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3896766464 bytes 2026-02-19T20:23:36.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3975941120 bytes 2026-02-19T20:23:36.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4026219520 bytes 2026-02-19T20:23:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4059393024 bytes 2026-02-19T20:23:36.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4089786368 bytes 2026-02-19T20:23:36.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4109733888 bytes 2026-02-19T20:23:36.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4131676160 bytes 2026-02-19T20:23:36.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4173836288 bytes 2026-02-19T20:23:36.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4209797120 bytes 2026-02-19T20:23:36.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4294811648 bytes 2026-02-19T20:23:37.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4342038528 bytes 2026-02-19T20:23:37.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4390847488 bytes 2026-02-19T20:23:37.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4413499392 bytes 2026-02-19T20:23:37.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4448836608 bytes 2026-02-19T20:23:37.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4521924608 bytes 2026-02-19T20:23:37.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4587348992 bytes 2026-02-19T20:23:37.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4619336704 bytes 2026-02-19T20:23:37.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4640056320 bytes 2026-02-19T20:23:37.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4689484800 bytes 2026-02-19T20:23:37.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4749531136 bytes 2026-02-19T20:23:37.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4805178368 bytes 2026-02-19T20:23:37.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4877563904 bytes 2026-02-19T20:23:37.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4901726208 bytes 2026-02-19T20:23:37.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4944680960 bytes 2026-02-19T20:23:37.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5031636992 bytes 2026-02-19T20:23:37.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5032240128 bytes 2026-02-19T20:23:38.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5042341888 bytes 2026-02-19T20:23:38.074 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2026-02-19T20:23:38.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2026-02-19T20:23:38.075 INFO:tasks.rbd_mirror_thrash:waiting for 4.36277240797968 secs before reviving daemons 2026-02-19T20:23:38.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:23:38.073+0000 7fa65c1d9640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 57565) UID: 1000 2026-02-19T20:23:38.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5107803136 bytes 2026-02-19T20:23:38.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5180722176 bytes 2026-02-19T20:23:38.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5237924864 bytes 2026-02-19T20:23:38.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5300487168 bytes 2026-02-19T20:23:38.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5335713792 bytes 2026-02-19T20:23:38.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5384377344 bytes 2026-02-19T20:23:38.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5405371392 bytes 2026-02-19T20:23:38.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5503368192 bytes 2026-02-19T20:23:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5520548864 bytes 2026-02-19T20:23:38.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5542729728 bytes 2026-02-19T20:23:38.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5579309056 bytes 2026-02-19T20:23:38.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5638477824 bytes 2026-02-19T20:23:38.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5670149120 bytes 2026-02-19T20:23:38.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5722395648 bytes 2026-02-19T20:23:38.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5763137536 bytes 2026-02-19T20:23:39.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5790475264 bytes 2026-02-19T20:23:39.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5804293120 bytes 2026-02-19T20:23:39.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5810827264 bytes 2026-02-19T20:23:39.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5862625280 bytes 2026-02-19T20:23:39.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5950738432 bytes 2026-02-19T20:23:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6002114560 bytes 2026-02-19T20:23:39.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6076126208 bytes 2026-02-19T20:23:39.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6091125760 bytes 2026-02-19T20:23:39.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6193112064 bytes 2026-02-19T20:23:39.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:23:39.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:23:39.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:23:39.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:39.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:23:39.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_30 2026-02-19T20:23:39.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_30 snapshot 128M 2026-02-19T20:23:39.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:39.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:39.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_30 128M 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:23:39.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:39.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:23:39.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:23:39.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:23:39.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:39.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_30 2026-02-19T20:23:39.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:23:39.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:39.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:23:39.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:39.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_30 snapshot 2026-02-19T20:23:39.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:39.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:39.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:23:39.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:39.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_30 snapshot 2026-02-19T20:23:40.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:23:40.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_30 2026-02-19T20:23:40.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_30': 2026-02-19T20:23:40.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:23:40.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:23:40.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:23:40.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 13cd3b716060 2026-02-19T20:23:40.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.13cd3b716060 2026-02-19T20:23:40.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:23:40.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:23:40.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:23:40.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:23:40.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:39 2026 2026-02-19T20:23:40.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:39 2026 2026-02-19T20:23:40.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:39 2026 2026-02-19T20:23:40.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:23:40.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:23:40.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 6e1e9027-c0b2-4632-8435-9250281efb2a 2026-02-19T20:23:40.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:23:40.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:23:40.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_30 2026-02-19T20:23:40.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:40.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:40.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:23:40.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:23:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=8 2026-02-19T20:23:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:23:40.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_30 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:23:40.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 614400 bytes 2026-02-19T20:23:40.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 44100608 bytes 2026-02-19T20:23:40.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 115173376 bytes 2026-02-19T20:23:40.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 154987520 bytes 2026-02-19T20:23:40.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 209410048 bytes 2026-02-19T20:23:41.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 236139520 bytes 2026-02-19T20:23:41.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 302555136 bytes 2026-02-19T20:23:41.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 356369408 bytes 2026-02-19T20:23:41.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 373847040 bytes 2026-02-19T20:23:41.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 417243136 bytes 2026-02-19T20:23:41.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 427880448 bytes 2026-02-19T20:23:41.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 443215872 bytes 2026-02-19T20:23:41.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 537127936 bytes 2026-02-19T20:23:41.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 580024320 bytes 2026-02-19T20:23:41.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 626777088 bytes 2026-02-19T20:23:41.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 676208640 bytes 2026-02-19T20:23:41.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 750191616 bytes 2026-02-19T20:23:41.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 800286720 bytes 2026-02-19T20:23:41.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 822055936 bytes 2026-02-19T20:23:41.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 907079680 bytes 2026-02-19T20:23:41.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 946051072 bytes 2026-02-19T20:23:42.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 987442176 bytes 2026-02-19T20:23:42.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1031000064 bytes 2026-02-19T20:23:42.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1099305984 bytes 2026-02-19T20:23:42.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1178228736 bytes 2026-02-19T20:23:42.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1225269248 bytes 2026-02-19T20:23:42.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1283063808 bytes 2026-02-19T20:23:42.438 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2026-02-19T20:23:42.438 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:23:42.438 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2026-02-19T20:23:42.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2026-02-19T20:23:42.439 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2026-02-19T20:23:42.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2026-02-19T20:23:42.441 INFO:tasks.rbd_mirror_thrash:waiting for 85.26192670784783 secs before thrashing 2026-02-19T20:23:42.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1307653120 bytes 2026-02-19T20:23:42.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1388367872 bytes 2026-02-19T20:23:42.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1418322944 bytes 2026-02-19T20:23:42.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1425597440 bytes 2026-02-19T20:23:42.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1462047744 bytes 2026-02-19T20:23:42.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1489652736 bytes 2026-02-19T20:23:42.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1545603072 bytes 2026-02-19T20:23:42.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1619405824 bytes 2026-02-19T20:23:42.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1658165248 bytes 2026-02-19T20:23:43.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1672292352 bytes 2026-02-19T20:23:43.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1708666880 bytes 2026-02-19T20:23:43.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1745753088 bytes 2026-02-19T20:23:43.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1773633536 bytes 2026-02-19T20:23:43.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1813986304 bytes 2026-02-19T20:23:43.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1824244736 bytes 2026-02-19T20:23:43.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1865014272 bytes 2026-02-19T20:23:43.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1918541824 bytes 2026-02-19T20:23:43.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1948493824 bytes 2026-02-19T20:23:43.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1971916800 bytes 2026-02-19T20:23:43.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2024079360 bytes 2026-02-19T20:23:43.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2033592320 bytes 2026-02-19T20:23:43.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2034170880 bytes 2026-02-19T20:23:43.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2088812544 bytes 2026-02-19T20:23:43.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2097332224 bytes 2026-02-19T20:23:43.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2099548160 bytes 2026-02-19T20:23:43.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2114795520 bytes 2026-02-19T20:23:43.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2142128128 bytes 2026-02-19T20:23:43.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2181746688 bytes 2026-02-19T20:23:44.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2222465024 bytes 2026-02-19T20:23:44.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2262024192 bytes 2026-02-19T20:23:44.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2301692928 bytes 2026-02-19T20:23:44.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2390626304 bytes 2026-02-19T20:23:44.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2475487232 bytes 2026-02-19T20:23:44.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2499097600 bytes 2026-02-19T20:23:44.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2513155072 bytes 2026-02-19T20:23:44.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2569910272 bytes 2026-02-19T20:23:44.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2599352320 bytes 2026-02-19T20:23:44.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2631288832 bytes 2026-02-19T20:23:44.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2741178368 bytes 2026-02-19T20:23:44.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2780834816 bytes 2026-02-19T20:23:45.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2821003264 bytes 2026-02-19T20:23:45.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2832625664 bytes 2026-02-19T20:23:45.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2855561216 bytes 2026-02-19T20:23:45.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2981819392 bytes 2026-02-19T20:23:45.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3015305216 bytes 2026-02-19T20:23:45.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3074587648 bytes 2026-02-19T20:23:45.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3123494912 bytes 2026-02-19T20:23:45.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3185660928 bytes 2026-02-19T20:23:45.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3237725184 bytes 2026-02-19T20:23:45.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3262195712 bytes 2026-02-19T20:23:45.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3341504512 bytes 2026-02-19T20:23:45.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3345120256 bytes 2026-02-19T20:23:45.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3456913408 bytes 2026-02-19T20:23:45.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3494632448 bytes 2026-02-19T20:23:46.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3573071872 bytes 2026-02-19T20:23:46.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3611188224 bytes 2026-02-19T20:23:46.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3661560832 bytes 2026-02-19T20:23:46.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3696690176 bytes 2026-02-19T20:23:46.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3724663808 bytes 2026-02-19T20:23:46.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3783771136 bytes 2026-02-19T20:23:46.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3820018688 bytes 2026-02-19T20:23:46.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3847619584 bytes 2026-02-19T20:23:46.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3893383168 bytes 2026-02-19T20:23:46.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3936309248 bytes 2026-02-19T20:23:46.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4080242688 bytes 2026-02-19T20:23:46.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4126130176 bytes 2026-02-19T20:23:46.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4138942464 bytes 2026-02-19T20:23:46.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4150816768 bytes 2026-02-19T20:23:46.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4194673664 bytes 2026-02-19T20:23:46.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4216440832 bytes 2026-02-19T20:23:46.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4220055552 bytes 2026-02-19T20:23:47.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4254422016 bytes 2026-02-19T20:23:47.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4280685568 bytes 2026-02-19T20:23:47.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4316689408 bytes 2026-02-19T20:23:47.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4370489344 bytes 2026-02-19T20:23:47.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4474167296 bytes 2026-02-19T20:23:47.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4489822208 bytes 2026-02-19T20:23:47.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4510070784 bytes 2026-02-19T20:23:47.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4548024320 bytes 2026-02-19T20:23:47.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4663990272 bytes 2026-02-19T20:23:47.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4727363584 bytes 2026-02-19T20:23:47.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4817924096 bytes 2026-02-19T20:23:47.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4836643840 bytes 2026-02-19T20:23:47.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4873594880 bytes 2026-02-19T20:23:47.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4957775872 bytes 2026-02-19T20:23:48.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5106432000 bytes 2026-02-19T20:23:48.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5126556672 bytes 2026-02-19T20:23:48.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5139105792 bytes 2026-02-19T20:23:48.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5157098496 bytes 2026-02-19T20:23:48.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5168748544 bytes 2026-02-19T20:23:48.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5200526336 bytes 2026-02-19T20:23:48.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5236508672 bytes 2026-02-19T20:23:48.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5288032256 bytes 2026-02-19T20:23:48.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5311003648 bytes 2026-02-19T20:23:48.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5348990976 bytes 2026-02-19T20:23:48.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5377987584 bytes 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_31 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_31 snapshot 128M 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:48.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_31 128M 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:23:48.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_31 2026-02-19T20:23:48.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:23:48.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:48.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:23:48.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:23:48.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_31 snapshot 2026-02-19T20:23:48.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:48.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:48.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:23:48.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:23:48.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_31 snapshot 2026-02-19T20:23:49.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:23:49.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_31 2026-02-19T20:23:49.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_31': 2026-02-19T20:23:49.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:23:49.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:23:49.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:23:49.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 13e2dbdc6f87 2026-02-19T20:23:49.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.13e2dbdc6f87 2026-02-19T20:23:49.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:23:49.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:23:49.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:23:49.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:23:49.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:48 2026 2026-02-19T20:23:49.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:48 2026 2026-02-19T20:23:49.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:48 2026 2026-02-19T20:23:49.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:23:49.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:23:49.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 41ae9cd0-a35e-4cfc-8eb3-abb6b3d0a94b 2026-02-19T20:23:49.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:23:49.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:23:49.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_31 2026-02-19T20:23:49.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:23:49.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:23:49.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:23:49.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:23:49.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:23:49.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:23:49.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_31 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:23:49.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 24818688 bytes 2026-02-19T20:23:49.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 35761152 bytes 2026-02-19T20:23:49.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 105121792 bytes 2026-02-19T20:23:49.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 164453376 bytes 2026-02-19T20:23:50.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 236313600 bytes 2026-02-19T20:23:50.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 257204224 bytes 2026-02-19T20:23:50.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5556115177725575) > weight (0.25) 2026-02-19T20:23:50.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6904300385073185) > weight (0.25) 2026-02-19T20:23:50.065 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2026-02-19T20:23:50.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2026-02-19T20:23:50.066 INFO:tasks.rbd_mirror_thrash:waiting for 0.945977441492013 secs before reviving daemons 2026-02-19T20:23:50.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:23:50.064+0000 7f42f662c640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 47856) UID: 1000 2026-02-19T20:23:50.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 322224128 bytes 2026-02-19T20:23:50.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 375568384 bytes 2026-02-19T20:23:50.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 396008448 bytes 2026-02-19T20:23:50.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 457049088 bytes 2026-02-19T20:23:50.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 463053824 bytes 2026-02-19T20:23:50.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 510236672 bytes 2026-02-19T20:23:50.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 585805824 bytes 2026-02-19T20:23:50.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 586408960 bytes 2026-02-19T20:23:50.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 587019264 bytes 2026-02-19T20:23:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 611840000 bytes 2026-02-19T20:23:50.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 616212480 bytes 2026-02-19T20:23:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 638032896 bytes 2026-02-19T20:23:50.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 661298176 bytes 2026-02-19T20:23:50.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 677623808 bytes 2026-02-19T20:23:50.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 702600192 bytes 2026-02-19T20:23:50.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 714540032 bytes 2026-02-19T20:23:50.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 821859328 bytes 2026-02-19T20:23:51.013 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2026-02-19T20:23:51.013 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:23:51.013 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2026-02-19T20:23:51.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2026-02-19T20:23:51.014 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2026-02-19T20:23:51.016 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2026-02-19T20:23:51.017 INFO:tasks.rbd_mirror_thrash:waiting for 98.29149955425473 secs before thrashing 2026-02-19T20:23:51.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 861669376 bytes 2026-02-19T20:23:51.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 919029760 bytes 2026-02-19T20:23:51.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 938776576 bytes 2026-02-19T20:23:51.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 989618176 bytes 2026-02-19T20:23:51.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1014881280 bytes 2026-02-19T20:23:51.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1042506752 bytes 2026-02-19T20:23:51.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1117664256 bytes 2026-02-19T20:23:51.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1162132480 bytes 2026-02-19T20:23:51.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1211252736 bytes 2026-02-19T20:23:51.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1243590656 bytes 2026-02-19T20:23:51.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1281362944 bytes 2026-02-19T20:23:51.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1293245440 bytes 2026-02-19T20:23:51.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1332651008 bytes 2026-02-19T20:23:51.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1350282240 bytes 2026-02-19T20:23:51.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1389443072 bytes 2026-02-19T20:23:51.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1438455808 bytes 2026-02-19T20:23:52.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1502919680 bytes 2026-02-19T20:23:52.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1554159616 bytes 2026-02-19T20:23:52.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1594540032 bytes 2026-02-19T20:23:52.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1595531264 bytes 2026-02-19T20:23:52.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1653998592 bytes 2026-02-19T20:23:52.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1706916864 bytes 2026-02-19T20:23:52.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1737716736 bytes 2026-02-19T20:23:52.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1770698752 bytes 2026-02-19T20:23:52.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1814537216 bytes 2026-02-19T20:23:52.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1895329792 bytes 2026-02-19T20:23:52.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1951504384 bytes 2026-02-19T20:23:52.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1988227072 bytes 2026-02-19T20:23:53.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2000264192 bytes 2026-02-19T20:23:53.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2040221696 bytes 2026-02-19T20:23:53.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2060774400 bytes 2026-02-19T20:23:53.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2089547776 bytes 2026-02-19T20:23:53.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2173534208 bytes 2026-02-19T20:23:53.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2205700096 bytes 2026-02-19T20:23:53.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2225260544 bytes 2026-02-19T20:23:53.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2274840576 bytes 2026-02-19T20:23:53.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2328870912 bytes 2026-02-19T20:23:53.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2380382208 bytes 2026-02-19T20:23:53.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2426562560 bytes 2026-02-19T20:23:53.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2469729280 bytes 2026-02-19T20:23:53.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2491828224 bytes 2026-02-19T20:23:53.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2574874624 bytes 2026-02-19T20:23:53.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2584001536 bytes 2026-02-19T20:23:53.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2611824640 bytes 2026-02-19T20:23:54.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2667838464 bytes 2026-02-19T20:23:54.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2748647424 bytes 2026-02-19T20:23:54.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2803342336 bytes 2026-02-19T20:23:54.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2826427392 bytes 2026-02-19T20:23:54.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2899203072 bytes 2026-02-19T20:23:54.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2929885184 bytes 2026-02-19T20:23:54.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3069150208 bytes 2026-02-19T20:23:54.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3108528128 bytes 2026-02-19T20:23:54.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3156709376 bytes 2026-02-19T20:23:54.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3187500032 bytes 2026-02-19T20:23:54.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3212403712 bytes 2026-02-19T20:23:54.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3258967040 bytes 2026-02-19T20:23:54.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3318131712 bytes 2026-02-19T20:23:54.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3352345600 bytes 2026-02-19T20:23:55.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3403717632 bytes 2026-02-19T20:23:55.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3443250176 bytes 2026-02-19T20:23:55.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3489249280 bytes 2026-02-19T20:23:55.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3546988544 bytes 2026-02-19T20:23:55.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3596822528 bytes 2026-02-19T20:23:55.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3650991104 bytes 2026-02-19T20:23:55.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3666717696 bytes 2026-02-19T20:23:55.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3685158912 bytes 2026-02-19T20:23:55.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3698101248 bytes 2026-02-19T20:23:55.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3763268608 bytes 2026-02-19T20:23:55.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3780023296 bytes 2026-02-19T20:23:55.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3836925952 bytes 2026-02-19T20:23:55.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3860924416 bytes 2026-02-19T20:23:55.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3916311552 bytes 2026-02-19T20:23:55.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3953911808 bytes 2026-02-19T20:23:55.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3964378112 bytes 2026-02-19T20:23:55.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3964990464 bytes 2026-02-19T20:23:56.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4041692160 bytes 2026-02-19T20:23:56.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4052476928 bytes 2026-02-19T20:23:56.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4099177472 bytes 2026-02-19T20:23:56.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4141884416 bytes 2026-02-19T20:23:56.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4219676672 bytes 2026-02-19T20:23:56.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4247071744 bytes 2026-02-19T20:23:56.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4284468224 bytes 2026-02-19T20:23:56.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4318899200 bytes 2026-02-19T20:23:56.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4434024448 bytes 2026-02-19T20:23:56.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4489619456 bytes 2026-02-19T20:23:56.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4557757440 bytes 2026-02-19T20:23:56.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4558375936 bytes 2026-02-19T20:23:56.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4621263872 bytes 2026-02-19T20:23:57.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4701907968 bytes 2026-02-19T20:23:57.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4715694080 bytes 2026-02-19T20:23:57.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4758899712 bytes 2026-02-19T20:23:57.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4796090368 bytes 2026-02-19T20:23:57.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4837634048 bytes 2026-02-19T20:23:57.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4882683904 bytes 2026-02-19T20:23:57.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4963786752 bytes 2026-02-19T20:23:57.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4988009472 bytes 2026-02-19T20:23:57.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5051450368 bytes 2026-02-19T20:23:57.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5122919424 bytes 2026-02-19T20:23:57.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5159424000 bytes 2026-02-19T20:23:57.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5205844992 bytes 2026-02-19T20:23:57.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5257778176 bytes 2026-02-19T20:23:57.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5284203520 bytes 2026-02-19T20:23:57.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5303818240 bytes 2026-02-19T20:23:58.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5360202752 bytes 2026-02-19T20:23:58.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5425574912 bytes 2026-02-19T20:23:58.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5539901440 bytes 2026-02-19T20:23:58.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5627479040 bytes 2026-02-19T20:23:58.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5661572096 bytes 2026-02-19T20:23:58.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5717524480 bytes 2026-02-19T20:23:58.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5779634176 bytes 2026-02-19T20:23:58.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5819451392 bytes 2026-02-19T20:23:58.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5858605056 bytes 2026-02-19T20:23:58.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5933152256 bytes 2026-02-19T20:23:58.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6008499200 bytes 2026-02-19T20:23:58.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6087548928 bytes 2026-02-19T20:23:58.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6163572736 bytes 2026-02-19T20:23:59.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6231129088 bytes 2026-02-19T20:23:59.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6276830208 bytes 2026-02-19T20:23:59.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6341591040 bytes 2026-02-19T20:23:59.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6371152896 bytes 2026-02-19T20:23:59.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6433559552 bytes 2026-02-19T20:23:59.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6510450688 bytes 2026-02-19T20:23:59.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6539947008 bytes 2026-02-19T20:23:59.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6560624640 bytes 2026-02-19T20:23:59.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6626191360 bytes 2026-02-19T20:23:59.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6657741824 bytes 2026-02-19T20:23:59.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6714557440 bytes 2026-02-19T20:23:59.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6802975744 bytes 2026-02-19T20:23:59.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6851404800 bytes 2026-02-19T20:23:59.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6890775552 bytes 2026-02-19T20:24:00.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6935739392 bytes 2026-02-19T20:24:00.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 7004690432 bytes 2026-02-19T20:24:00.158 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:24:00.155+0000 7f5bf91cc640 -1 reset not still connected to 0x7f5f2b0 2026-02-19T20:24:00.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7008768000 bytes 2026-02-19T20:24:00.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7030359040 bytes 2026-02-19T20:24:00.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7036130304 bytes 2026-02-19T20:24:00.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7059131392 bytes 2026-02-19T20:24:00.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7192870912 bytes 2026-02-19T20:24:00.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7238667264 bytes 2026-02-19T20:24:00.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7255036928 bytes 2026-02-19T20:24:00.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7325690880 bytes 2026-02-19T20:24:00.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7347720192 bytes 2026-02-19T20:24:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7380676608 bytes 2026-02-19T20:24:00.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7391674368 bytes 2026-02-19T20:24:00.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7445345280 bytes 2026-02-19T20:24:00.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7516900352 bytes 2026-02-19T20:24:00.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7559215104 bytes 2026-02-19T20:24:01.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7602750464 bytes 2026-02-19T20:24:01.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7659353088 bytes 2026-02-19T20:24:01.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7743465472 bytes 2026-02-19T20:24:01.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7744070656 bytes 2026-02-19T20:24:01.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7844540416 bytes 2026-02-19T20:24:01.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7845161984 bytes 2026-02-19T20:24:01.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7873660928 bytes 2026-02-19T20:24:01.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:24:01.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:24:01.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:24:01.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:01.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:24:01.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_32 2026-02-19T20:24:01.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_32 snapshot 128M 2026-02-19T20:24:01.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_32 128M 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:01.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:01.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:01.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:24:01.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:01.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:24:01.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:24:01.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:24:01.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:01.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_32 2026-02-19T20:24:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:24:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:01.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:24:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_32 snapshot 2026-02-19T20:24:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:24:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:01.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_32 snapshot 2026-02-19T20:24:02.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:24:02.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_32 2026-02-19T20:24:02.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_32': 2026-02-19T20:24:02.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:24:02.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:24:02.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:24:02.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 13f47ac2f76a 2026-02-19T20:24:02.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.13f47ac2f76a 2026-02-19T20:24:02.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:24:02.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:24:02.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:24:02.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:24:02.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:01 2026 2026-02-19T20:24:02.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:01 2026 2026-02-19T20:24:02.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:01 2026 2026-02-19T20:24:02.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:24:02.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:24:02.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 8f9355c3-ffc8-4071-aaf4-f8356e2ec162 2026-02-19T20:24:02.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:24:02.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:24:02.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_32 2026-02-19T20:24:02.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:02.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:02.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:24:02.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:24:02.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=11 2026-02-19T20:24:02.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:24:02.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_32 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:24:02.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 19066880 bytes 2026-02-19T20:24:02.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 34406400 bytes 2026-02-19T20:24:02.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 101405696 bytes 2026-02-19T20:24:03.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 161762304 bytes 2026-02-19T20:24:03.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 200072192 bytes 2026-02-19T20:24:03.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 220388352 bytes 2026-02-19T20:24:03.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 275209216 bytes 2026-02-19T20:24:03.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 314014720 bytes 2026-02-19T20:24:03.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 363235328 bytes 2026-02-19T20:24:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 413428736 bytes 2026-02-19T20:24:03.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 436892672 bytes 2026-02-19T20:24:03.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 506192896 bytes 2026-02-19T20:24:03.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 514620416 bytes 2026-02-19T20:24:03.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 563073024 bytes 2026-02-19T20:24:03.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 621467648 bytes 2026-02-19T20:24:03.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 656946176 bytes 2026-02-19T20:24:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 676550656 bytes 2026-02-19T20:24:03.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 696916992 bytes 2026-02-19T20:24:03.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 728768512 bytes 2026-02-19T20:24:04.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 750109696 bytes 2026-02-19T20:24:04.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 750721024 bytes 2026-02-19T20:24:04.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 773975040 bytes 2026-02-19T20:24:04.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 799129600 bytes 2026-02-19T20:24:04.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 842027008 bytes 2026-02-19T20:24:04.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 870348800 bytes 2026-02-19T20:24:04.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 891871232 bytes 2026-02-19T20:24:04.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 914516992 bytes 2026-02-19T20:24:04.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 991418368 bytes 2026-02-19T20:24:04.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 992839680 bytes 2026-02-19T20:24:04.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1003386880 bytes 2026-02-19T20:24:04.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1041654784 bytes 2026-02-19T20:24:04.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1078339584 bytes 2026-02-19T20:24:04.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1113112576 bytes 2026-02-19T20:24:04.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1180292096 bytes 2026-02-19T20:24:04.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1233615872 bytes 2026-02-19T20:24:04.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1270164480 bytes 2026-02-19T20:24:04.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1298008064 bytes 2026-02-19T20:24:04.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1382986752 bytes 2026-02-19T20:24:05.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1415689216 bytes 2026-02-19T20:24:05.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1482509312 bytes 2026-02-19T20:24:05.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1508534272 bytes 2026-02-19T20:24:05.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1523412992 bytes 2026-02-19T20:24:05.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1542579200 bytes 2026-02-19T20:24:05.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1572082688 bytes 2026-02-19T20:24:05.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1582416896 bytes 2026-02-19T20:24:05.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1621393408 bytes 2026-02-19T20:24:05.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1653939200 bytes 2026-02-19T20:24:05.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1705395200 bytes 2026-02-19T20:24:05.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1771022336 bytes 2026-02-19T20:24:05.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1830364160 bytes 2026-02-19T20:24:05.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1932816384 bytes 2026-02-19T20:24:05.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 1935442944 bytes 2026-02-19T20:24:05.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2042588160 bytes 2026-02-19T20:24:05.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2063031296 bytes 2026-02-19T20:24:06.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2115510272 bytes 2026-02-19T20:24:06.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2117664768 bytes 2026-02-19T20:24:06.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2134970368 bytes 2026-02-19T20:24:06.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2158142464 bytes 2026-02-19T20:24:06.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2256909312 bytes 2026-02-19T20:24:06.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2331369472 bytes 2026-02-19T20:24:06.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2365855744 bytes 2026-02-19T20:24:06.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2366474240 bytes 2026-02-19T20:24:06.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2411452416 bytes 2026-02-19T20:24:06.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2440578048 bytes 2026-02-19T20:24:06.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2454441984 bytes 2026-02-19T20:24:06.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2518405120 bytes 2026-02-19T20:24:06.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2581451776 bytes 2026-02-19T20:24:06.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2623002624 bytes 2026-02-19T20:24:06.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2684868608 bytes 2026-02-19T20:24:06.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2705976320 bytes 2026-02-19T20:24:07.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2766983168 bytes 2026-02-19T20:24:07.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2801836032 bytes 2026-02-19T20:24:07.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2847633408 bytes 2026-02-19T20:24:07.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2861424640 bytes 2026-02-19T20:24:07.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2884049920 bytes 2026-02-19T20:24:07.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2892708864 bytes 2026-02-19T20:24:07.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 2916942848 bytes 2026-02-19T20:24:07.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 2988125184 bytes 2026-02-19T20:24:07.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3050008576 bytes 2026-02-19T20:24:07.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3108914176 bytes 2026-02-19T20:24:07.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3127335936 bytes 2026-02-19T20:24:07.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3190785024 bytes 2026-02-19T20:24:07.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3252212736 bytes 2026-02-19T20:24:07.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3293281280 bytes 2026-02-19T20:24:07.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3342740480 bytes 2026-02-19T20:24:07.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3383345152 bytes 2026-02-19T20:24:07.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3395069952 bytes 2026-02-19T20:24:08.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3427009536 bytes 2026-02-19T20:24:08.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3501477888 bytes 2026-02-19T20:24:08.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3508757504 bytes 2026-02-19T20:24:08.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3530144768 bytes 2026-02-19T20:24:08.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3564576768 bytes 2026-02-19T20:24:08.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3618818048 bytes 2026-02-19T20:24:08.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3633415168 bytes 2026-02-19T20:24:08.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3688872960 bytes 2026-02-19T20:24:08.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3747780608 bytes 2026-02-19T20:24:08.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3789180928 bytes 2026-02-19T20:24:08.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3893727232 bytes 2026-02-19T20:24:08.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3947456512 bytes 2026-02-19T20:24:08.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 3999418368 bytes 2026-02-19T20:24:08.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4026074112 bytes 2026-02-19T20:24:08.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4066458624 bytes 2026-02-19T20:24:08.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4074599424 bytes 2026-02-19T20:24:09.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4176240640 bytes 2026-02-19T20:24:09.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4218168320 bytes 2026-02-19T20:24:09.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4227915776 bytes 2026-02-19T20:24:09.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4252232704 bytes 2026-02-19T20:24:09.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4311292928 bytes 2026-02-19T20:24:09.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4333212672 bytes 2026-02-19T20:24:09.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4399849472 bytes 2026-02-19T20:24:09.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4444375040 bytes 2026-02-19T20:24:09.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4494994432 bytes 2026-02-19T20:24:09.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4576167936 bytes 2026-02-19T20:24:09.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4590360576 bytes 2026-02-19T20:24:09.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4673816576 bytes 2026-02-19T20:24:09.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4679130112 bytes 2026-02-19T20:24:09.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4706689024 bytes 2026-02-19T20:24:09.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4742598656 bytes 2026-02-19T20:24:09.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4761222144 bytes 2026-02-19T20:24:10.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4853547008 bytes 2026-02-19T20:24:10.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4884488192 bytes 2026-02-19T20:24:10.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4914679808 bytes 2026-02-19T20:24:10.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4934139904 bytes 2026-02-19T20:24:10.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4953180160 bytes 2026-02-19T20:24:10.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5005629440 bytes 2026-02-19T20:24:10.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5064136704 bytes 2026-02-19T20:24:10.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5106762752 bytes 2026-02-19T20:24:10.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5147703296 bytes 2026-02-19T20:24:10.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5223990272 bytes 2026-02-19T20:24:10.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5246188544 bytes 2026-02-19T20:24:10.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5329436672 bytes 2026-02-19T20:24:10.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5369449472 bytes 2026-02-19T20:24:10.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5429633024 bytes 2026-02-19T20:24:11.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5487329280 bytes 2026-02-19T20:24:11.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5520174080 bytes 2026-02-19T20:24:11.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5565146112 bytes 2026-02-19T20:24:11.196 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5609279488 bytes 2026-02-19T20:24:11.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5634897920 bytes 2026-02-19T20:24:11.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5654964224 bytes 2026-02-19T20:24:11.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5698118656 bytes 2026-02-19T20:24:11.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5745826816 bytes 2026-02-19T20:24:11.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5755665408 bytes 2026-02-19T20:24:11.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5789601792 bytes 2026-02-19T20:24:11.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5829653504 bytes 2026-02-19T20:24:11.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5884766208 bytes 2026-02-19T20:24:11.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5948090368 bytes 2026-02-19T20:24:11.762 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5956590592 bytes 2026-02-19T20:24:11.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6067431424 bytes 2026-02-19T20:24:11.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6109807616 bytes 2026-02-19T20:24:11.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6143655936 bytes 2026-02-19T20:24:12.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6177618944 bytes 2026-02-19T20:24:12.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6193208320 bytes 2026-02-19T20:24:12.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6230579200 bytes 2026-02-19T20:24:12.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6269764608 bytes 2026-02-19T20:24:12.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6344023040 bytes 2026-02-19T20:24:12.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6391158784 bytes 2026-02-19T20:24:12.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6410568704 bytes 2026-02-19T20:24:12.439 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6442752000 bytes 2026-02-19T20:24:12.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6490204160 bytes 2026-02-19T20:24:12.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6526743552 bytes 2026-02-19T20:24:12.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6568364032 bytes 2026-02-19T20:24:12.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6576753664 bytes 2026-02-19T20:24:12.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6588070912 bytes 2026-02-19T20:24:12.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6623870976 bytes 2026-02-19T20:24:12.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6715062272 bytes 2026-02-19T20:24:12.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6777724928 bytes 2026-02-19T20:24:12.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6807995392 bytes 2026-02-19T20:24:13.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6906943488 bytes 2026-02-19T20:24:13.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6974799872 bytes 2026-02-19T20:24:13.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7004925952 bytes 2026-02-19T20:24:13.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7063498752 bytes 2026-02-19T20:24:13.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7104903168 bytes 2026-02-19T20:24:13.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7137390592 bytes 2026-02-19T20:24:13.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7137988608 bytes 2026-02-19T20:24:13.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7159127040 bytes 2026-02-19T20:24:13.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7223856128 bytes 2026-02-19T20:24:13.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7241705472 bytes 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_33 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_33 snapshot 128M 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_33 128M 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_33 2026-02-19T20:24:13.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:24:13.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:13.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:24:13.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:13.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_33 snapshot 2026-02-19T20:24:13.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:13.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:13.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:24:13.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:13.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_33 snapshot 2026-02-19T20:24:14.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:24:14.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_33 2026-02-19T20:24:14.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_33': 2026-02-19T20:24:14.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:24:14.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:24:14.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:24:14.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1400dc750059 2026-02-19T20:24:14.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1400dc750059 2026-02-19T20:24:14.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:24:14.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:24:14.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:24:14.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:24:14.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:13 2026 2026-02-19T20:24:14.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:13 2026 2026-02-19T20:24:14.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:13 2026 2026-02-19T20:24:14.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:24:14.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:24:14.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 4410c659-b877-49f7-b76c-9e1e17ec8cee 2026-02-19T20:24:14.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:24:14.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:24:14.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_33 2026-02-19T20:24:14.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:14.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:14.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:24:14.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:24:14.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=9 2026-02-19T20:24:14.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:24:14.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_33 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:24:14.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 28044288 bytes 2026-02-19T20:24:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 120321024 bytes 2026-02-19T20:24:14.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 210006016 bytes 2026-02-19T20:24:14.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 214064128 bytes 2026-02-19T20:24:14.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 227053568 bytes 2026-02-19T20:24:14.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 300530688 bytes 2026-02-19T20:24:15.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 350470144 bytes 2026-02-19T20:24:15.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 447499264 bytes 2026-02-19T20:24:15.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 502249472 bytes 2026-02-19T20:24:15.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 533178368 bytes 2026-02-19T20:24:15.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 551593984 bytes 2026-02-19T20:24:15.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 577040384 bytes 2026-02-19T20:24:15.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 620577792 bytes 2026-02-19T20:24:15.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 621182976 bytes 2026-02-19T20:24:15.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 676280320 bytes 2026-02-19T20:24:15.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 694673408 bytes 2026-02-19T20:24:15.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 714120192 bytes 2026-02-19T20:24:15.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 758217728 bytes 2026-02-19T20:24:15.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 843782144 bytes 2026-02-19T20:24:15.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 855959552 bytes 2026-02-19T20:24:15.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 869766144 bytes 2026-02-19T20:24:15.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 890442752 bytes 2026-02-19T20:24:15.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 901511168 bytes 2026-02-19T20:24:16.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 924124160 bytes 2026-02-19T20:24:16.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 975934464 bytes 2026-02-19T20:24:16.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1037105152 bytes 2026-02-19T20:24:16.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1107564544 bytes 2026-02-19T20:24:16.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1136359424 bytes 2026-02-19T20:24:16.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1170277376 bytes 2026-02-19T20:24:16.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1237644288 bytes 2026-02-19T20:24:16.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1273135104 bytes 2026-02-19T20:24:16.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1309600768 bytes 2026-02-19T20:24:16.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1342654464 bytes 2026-02-19T20:24:16.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1367127040 bytes 2026-02-19T20:24:16.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1429201920 bytes 2026-02-19T20:24:16.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1447071744 bytes 2026-02-19T20:24:16.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1509690368 bytes 2026-02-19T20:24:16.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1524817920 bytes 2026-02-19T20:24:16.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1620029440 bytes 2026-02-19T20:24:17.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1699859456 bytes 2026-02-19T20:24:17.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1789194240 bytes 2026-02-19T20:24:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1832093696 bytes 2026-02-19T20:24:17.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1911638016 bytes 2026-02-19T20:24:17.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2003337216 bytes 2026-02-19T20:24:17.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2044339200 bytes 2026-02-19T20:24:17.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2117855232 bytes 2026-02-19T20:24:17.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2128530432 bytes 2026-02-19T20:24:17.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2133993472 bytes 2026-02-19T20:24:17.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2190524416 bytes 2026-02-19T20:24:17.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2226197504 bytes 2026-02-19T20:24:17.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2256915456 bytes 2026-02-19T20:24:17.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2294174720 bytes 2026-02-19T20:24:17.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2334073856 bytes 2026-02-19T20:24:17.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2353825792 bytes 2026-02-19T20:24:17.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2392188928 bytes 2026-02-19T20:24:18.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2454057984 bytes 2026-02-19T20:24:18.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2454681600 bytes 2026-02-19T20:24:18.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2485352448 bytes 2026-02-19T20:24:18.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2574422016 bytes 2026-02-19T20:24:18.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2623560704 bytes 2026-02-19T20:24:18.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2705434624 bytes 2026-02-19T20:24:18.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2757383168 bytes 2026-02-19T20:24:18.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2792841216 bytes 2026-02-19T20:24:18.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2834130944 bytes 2026-02-19T20:24:18.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2883090432 bytes 2026-02-19T20:24:18.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2899911680 bytes 2026-02-19T20:24:18.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2987571200 bytes 2026-02-19T20:24:18.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2998808576 bytes 2026-02-19T20:24:18.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3021915136 bytes 2026-02-19T20:24:18.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3067436032 bytes 2026-02-19T20:24:18.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3131579392 bytes 2026-02-19T20:24:19.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3143669760 bytes 2026-02-19T20:24:19.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3145224192 bytes 2026-02-19T20:24:19.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3182948352 bytes 2026-02-19T20:24:19.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3217086464 bytes 2026-02-19T20:24:19.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3227700224 bytes 2026-02-19T20:24:19.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3286727680 bytes 2026-02-19T20:24:19.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3381604352 bytes 2026-02-19T20:24:19.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3448367104 bytes 2026-02-19T20:24:19.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3478659072 bytes 2026-02-19T20:24:19.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3542685696 bytes 2026-02-19T20:24:19.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3568327680 bytes 2026-02-19T20:24:19.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3629471744 bytes 2026-02-19T20:24:19.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3650946048 bytes 2026-02-19T20:24:19.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3699312640 bytes 2026-02-19T20:24:19.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3732127744 bytes 2026-02-19T20:24:19.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3802995712 bytes 2026-02-19T20:24:20.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3873134592 bytes 2026-02-19T20:24:20.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3911523328 bytes 2026-02-19T20:24:20.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3984199680 bytes 2026-02-19T20:24:20.196 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3994249216 bytes 2026-02-19T20:24:20.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4017686528 bytes 2026-02-19T20:24:20.308 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:24:20.305+0000 7f5bf91cc640 -1 reset not still connected to 0xc092d00 2026-02-19T20:24:20.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4074542080 bytes 2026-02-19T20:24:20.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4085047296 bytes 2026-02-19T20:24:20.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4104747008 bytes 2026-02-19T20:24:20.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4112476160 bytes 2026-02-19T20:24:20.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4201287680 bytes 2026-02-19T20:24:20.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4273245184 bytes 2026-02-19T20:24:20.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4319454208 bytes 2026-02-19T20:24:20.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4327522304 bytes 2026-02-19T20:24:20.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4392020992 bytes 2026-02-19T20:24:21.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4428110848 bytes 2026-02-19T20:24:21.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4493775872 bytes 2026-02-19T20:24:21.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4551885824 bytes 2026-02-19T20:24:21.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4572247040 bytes 2026-02-19T20:24:21.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4658529280 bytes 2026-02-19T20:24:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4687430656 bytes 2026-02-19T20:24:21.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4745034752 bytes 2026-02-19T20:24:21.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4745652224 bytes 2026-02-19T20:24:21.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4789656576 bytes 2026-02-19T20:24:21.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4852496384 bytes 2026-02-19T20:24:21.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4875981824 bytes 2026-02-19T20:24:21.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4908892160 bytes 2026-02-19T20:24:21.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4927834112 bytes 2026-02-19T20:24:22.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4977116160 bytes 2026-02-19T20:24:22.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5009619968 bytes 2026-02-19T20:24:22.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5024218112 bytes 2026-02-19T20:24:22.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5077308416 bytes 2026-02-19T20:24:22.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5103288320 bytes 2026-02-19T20:24:22.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5155899392 bytes 2026-02-19T20:24:22.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5208961024 bytes 2026-02-19T20:24:22.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5285138432 bytes 2026-02-19T20:24:22.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5325405184 bytes 2026-02-19T20:24:22.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5362323456 bytes 2026-02-19T20:24:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5406115840 bytes 2026-02-19T20:24:22.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5462541312 bytes 2026-02-19T20:24:22.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5464856576 bytes 2026-02-19T20:24:22.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5499177984 bytes 2026-02-19T20:24:22.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5552052224 bytes 2026-02-19T20:24:22.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5606917120 bytes 2026-02-19T20:24:23.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5636643840 bytes 2026-02-19T20:24:23.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5667119104 bytes 2026-02-19T20:24:23.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5684880384 bytes 2026-02-19T20:24:23.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5701004288 bytes 2026-02-19T20:24:23.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5773583360 bytes 2026-02-19T20:24:23.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5851793408 bytes 2026-02-19T20:24:23.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5904340992 bytes 2026-02-19T20:24:23.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5921042432 bytes 2026-02-19T20:24:23.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5994023936 bytes 2026-02-19T20:24:23.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6004375552 bytes 2026-02-19T20:24:23.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:24:23.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:24:23.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:24:23.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:23.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:24:23.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_34 2026-02-19T20:24:23.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_34 snapshot 128M 2026-02-19T20:24:23.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:23.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:23.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_34 128M 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:23.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:24:23.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:23.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:24:23.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:24:23.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:24:23.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:23.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_34 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_34 snapshot 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:23.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_34 snapshot 2026-02-19T20:24:24.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:24:24.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_34 2026-02-19T20:24:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_34': 2026-02-19T20:24:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:24:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:24:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:24:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 140c57e0cd31 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.140c57e0cd31 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:23 2026 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:23 2026 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:23 2026 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:24:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: e3994cd0-fb3f-4a15-bc7e-aa4c9bd26c87 2026-02-19T20:24:24.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:24:24.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:24:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_34 2026-02-19T20:24:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:24:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:24:24.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=6 2026-02-19T20:24:24.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:24:24.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_34 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:24:24.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 57594880 bytes 2026-02-19T20:24:24.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 99611648 bytes 2026-02-19T20:24:24.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 150692864 bytes 2026-02-19T20:24:24.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 174172160 bytes 2026-02-19T20:24:24.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 207710208 bytes 2026-02-19T20:24:25.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 232060928 bytes 2026-02-19T20:24:25.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 238367744 bytes 2026-02-19T20:24:25.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 245303296 bytes 2026-02-19T20:24:25.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 323916800 bytes 2026-02-19T20:24:25.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 342606848 bytes 2026-02-19T20:24:25.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 391476224 bytes 2026-02-19T20:24:25.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 422241280 bytes 2026-02-19T20:24:25.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 483635200 bytes 2026-02-19T20:24:25.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 536016896 bytes 2026-02-19T20:24:25.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 563557376 bytes 2026-02-19T20:24:25.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 618068992 bytes 2026-02-19T20:24:25.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 654707712 bytes 2026-02-19T20:24:25.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 675707904 bytes 2026-02-19T20:24:25.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 712040448 bytes 2026-02-19T20:24:25.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 751111168 bytes 2026-02-19T20:24:25.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 802009088 bytes 2026-02-19T20:24:25.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 826693632 bytes 2026-02-19T20:24:26.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 876534784 bytes 2026-02-19T20:24:26.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 904709120 bytes 2026-02-19T20:24:26.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 950017024 bytes 2026-02-19T20:24:26.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 965513216 bytes 2026-02-19T20:24:26.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1048797184 bytes 2026-02-19T20:24:26.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1074173952 bytes 2026-02-19T20:24:26.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1087281152 bytes 2026-02-19T20:24:26.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1136181248 bytes 2026-02-19T20:24:26.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1200727040 bytes 2026-02-19T20:24:26.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1225598976 bytes 2026-02-19T20:24:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1330640896 bytes 2026-02-19T20:24:26.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1362855936 bytes 2026-02-19T20:24:26.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1415116800 bytes 2026-02-19T20:24:26.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1456512000 bytes 2026-02-19T20:24:26.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1488136192 bytes 2026-02-19T20:24:27.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1520273408 bytes 2026-02-19T20:24:27.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1520884736 bytes 2026-02-19T20:24:27.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1573223424 bytes 2026-02-19T20:24:27.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1587971072 bytes 2026-02-19T20:24:27.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1616335872 bytes 2026-02-19T20:24:27.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1654391808 bytes 2026-02-19T20:24:27.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1738637312 bytes 2026-02-19T20:24:27.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1774025728 bytes 2026-02-19T20:24:27.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1834262528 bytes 2026-02-19T20:24:27.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1851452416 bytes 2026-02-19T20:24:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1916741632 bytes 2026-02-19T20:24:27.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1939405824 bytes 2026-02-19T20:24:27.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1939997696 bytes 2026-02-19T20:24:27.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1974981632 bytes 2026-02-19T20:24:27.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2000638976 bytes 2026-02-19T20:24:27.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2011335680 bytes 2026-02-19T20:24:28.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2093464576 bytes 2026-02-19T20:24:28.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2107175936 bytes 2026-02-19T20:24:28.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2130934784 bytes 2026-02-19T20:24:28.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2194264064 bytes 2026-02-19T20:24:28.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2201027584 bytes 2026-02-19T20:24:28.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2243863552 bytes 2026-02-19T20:24:28.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2300392448 bytes 2026-02-19T20:24:28.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2327822336 bytes 2026-02-19T20:24:28.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2363258880 bytes 2026-02-19T20:24:28.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2365480960 bytes 2026-02-19T20:24:28.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2398520320 bytes 2026-02-19T20:24:28.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2444139520 bytes 2026-02-19T20:24:28.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2486151168 bytes 2026-02-19T20:24:28.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2561458176 bytes 2026-02-19T20:24:28.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2633482240 bytes 2026-02-19T20:24:28.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2672636928 bytes 2026-02-19T20:24:29.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2718091264 bytes 2026-02-19T20:24:29.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2739336192 bytes 2026-02-19T20:24:29.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2797694976 bytes 2026-02-19T20:24:29.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2875432960 bytes 2026-02-19T20:24:29.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2905978880 bytes 2026-02-19T20:24:29.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2958098432 bytes 2026-02-19T20:24:29.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3014717440 bytes 2026-02-19T20:24:29.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3025101824 bytes 2026-02-19T20:24:29.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3029615616 bytes 2026-02-19T20:24:29.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3059859456 bytes 2026-02-19T20:24:29.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3081586688 bytes 2026-02-19T20:24:29.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3093792768 bytes 2026-02-19T20:24:29.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3108130816 bytes 2026-02-19T20:24:29.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3138124800 bytes 2026-02-19T20:24:29.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3217761280 bytes 2026-02-19T20:24:29.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3283851264 bytes 2026-02-19T20:24:29.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3313526784 bytes 2026-02-19T20:24:30.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3335783424 bytes 2026-02-19T20:24:30.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3392813056 bytes 2026-02-19T20:24:30.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3444040704 bytes 2026-02-19T20:24:30.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3491670016 bytes 2026-02-19T20:24:30.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3584132096 bytes 2026-02-19T20:24:30.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3587215360 bytes 2026-02-19T20:24:30.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3633362944 bytes 2026-02-19T20:24:30.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3675722752 bytes 2026-02-19T20:24:30.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:24:30.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:24:30.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_35 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_35 snapshot 128M 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:30.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_35 128M 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:24:30.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:30.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_35 2026-02-19T20:24:30.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:24:30.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:30.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:24:30.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:30.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_35 snapshot 2026-02-19T20:24:30.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:30.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:30.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:24:30.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:30.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_35 snapshot 2026-02-19T20:24:31.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:24:31.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_35 2026-02-19T20:24:31.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_35': 2026-02-19T20:24:31.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:24:31.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:24:31.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:24:31.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 141859b42726 2026-02-19T20:24:31.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.141859b42726 2026-02-19T20:24:31.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:24:31.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:24:31.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:24:31.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:24:31.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:30 2026 2026-02-19T20:24:31.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:30 2026 2026-02-19T20:24:31.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:30 2026 2026-02-19T20:24:31.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:24:31.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:24:31.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 1d72d7ff-8dde-42b4-9811-16a6c4bba739 2026-02-19T20:24:31.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:24:31.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:24:31.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_35 2026-02-19T20:24:31.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:31.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:31.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:24:31.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:24:31.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=7 2026-02-19T20:24:31.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:24:31.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_35 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:24:31.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 75859968 bytes 2026-02-19T20:24:31.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 122825728 bytes 2026-02-19T20:24:31.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 189718528 bytes 2026-02-19T20:24:31.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 280259584 bytes 2026-02-19T20:24:32.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 283227136 bytes 2026-02-19T20:24:32.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 351939584 bytes 2026-02-19T20:24:32.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 380744704 bytes 2026-02-19T20:24:32.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 429506560 bytes 2026-02-19T20:24:32.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 480564224 bytes 2026-02-19T20:24:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 542787584 bytes 2026-02-19T20:24:32.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 654404608 bytes 2026-02-19T20:24:32.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 732910592 bytes 2026-02-19T20:24:32.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 794850304 bytes 2026-02-19T20:24:32.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 796827648 bytes 2026-02-19T20:24:32.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 799009792 bytes 2026-02-19T20:24:32.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 863897600 bytes 2026-02-19T20:24:32.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 904552448 bytes 2026-02-19T20:24:32.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 943420416 bytes 2026-02-19T20:24:32.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 958102528 bytes 2026-02-19T20:24:33.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 1015793664 bytes 2026-02-19T20:24:33.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1016416256 bytes 2026-02-19T20:24:33.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1077347328 bytes 2026-02-19T20:24:33.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1099226112 bytes 2026-02-19T20:24:33.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1138237440 bytes 2026-02-19T20:24:33.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1274702848 bytes 2026-02-19T20:24:33.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1314998272 bytes 2026-02-19T20:24:33.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1396856832 bytes 2026-02-19T20:24:33.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1437649920 bytes 2026-02-19T20:24:33.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1470707712 bytes 2026-02-19T20:24:33.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1483065344 bytes 2026-02-19T20:24:33.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1577214976 bytes 2026-02-19T20:24:33.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1610416128 bytes 2026-02-19T20:24:33.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1700501504 bytes 2026-02-19T20:24:34.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1729896448 bytes 2026-02-19T20:24:34.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1846677504 bytes 2026-02-19T20:24:34.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1885627392 bytes 2026-02-19T20:24:34.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1957275648 bytes 2026-02-19T20:24:34.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 2026425344 bytes 2026-02-19T20:24:34.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 2067501056 bytes 2026-02-19T20:24:34.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 2087449600 bytes 2026-02-19T20:24:34.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 2118309888 bytes 2026-02-19T20:24:34.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 2148134912 bytes 2026-02-19T20:24:34.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2180582400 bytes 2026-02-19T20:24:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2226313216 bytes 2026-02-19T20:24:34.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2304056320 bytes 2026-02-19T20:24:34.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2361368576 bytes 2026-02-19T20:24:34.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2412710912 bytes 2026-02-19T20:24:35.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2504175616 bytes 2026-02-19T20:24:35.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2584168448 bytes 2026-02-19T20:24:35.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2635122688 bytes 2026-02-19T20:24:35.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2669593600 bytes 2026-02-19T20:24:35.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2737228800 bytes 2026-02-19T20:24:35.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2772743168 bytes 2026-02-19T20:24:35.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2794363904 bytes 2026-02-19T20:24:35.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2833903616 bytes 2026-02-19T20:24:35.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2891544576 bytes 2026-02-19T20:24:35.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2906506240 bytes 2026-02-19T20:24:35.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2971016192 bytes 2026-02-19T20:24:35.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2984479744 bytes 2026-02-19T20:24:35.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2989383680 bytes 2026-02-19T20:24:35.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 3026150400 bytes 2026-02-19T20:24:35.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 3079057408 bytes 2026-02-19T20:24:35.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 3106850816 bytes 2026-02-19T20:24:36.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 3136746496 bytes 2026-02-19T20:24:36.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3146017792 bytes 2026-02-19T20:24:36.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3210452992 bytes 2026-02-19T20:24:36.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3247267840 bytes 2026-02-19T20:24:36.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3299884032 bytes 2026-02-19T20:24:36.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3372873728 bytes 2026-02-19T20:24:36.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3437327360 bytes 2026-02-19T20:24:36.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3466894336 bytes 2026-02-19T20:24:36.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3498360832 bytes 2026-02-19T20:24:36.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3528349696 bytes 2026-02-19T20:24:36.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3569067008 bytes 2026-02-19T20:24:36.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3592070144 bytes 2026-02-19T20:24:36.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3617291264 bytes 2026-02-19T20:24:36.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3665581056 bytes 2026-02-19T20:24:36.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3666190336 bytes 2026-02-19T20:24:36.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3700615168 bytes 2026-02-19T20:24:37.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3792399360 bytes 2026-02-19T20:24:37.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3865017344 bytes 2026-02-19T20:24:37.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3881737216 bytes 2026-02-19T20:24:37.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3934572544 bytes 2026-02-19T20:24:37.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3986387968 bytes 2026-02-19T20:24:37.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 4059000832 bytes 2026-02-19T20:24:37.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 4092290048 bytes 2026-02-19T20:24:37.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4154179584 bytes 2026-02-19T20:24:37.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4188823552 bytes 2026-02-19T20:24:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4241863680 bytes 2026-02-19T20:24:37.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4253995008 bytes 2026-02-19T20:24:37.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4299514880 bytes 2026-02-19T20:24:37.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4308346880 bytes 2026-02-19T20:24:37.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4363747328 bytes 2026-02-19T20:24:37.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4405626880 bytes 2026-02-19T20:24:38.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4449702912 bytes 2026-02-19T20:24:38.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4524957696 bytes 2026-02-19T20:24:38.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4578057216 bytes 2026-02-19T20:24:38.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4594220032 bytes 2026-02-19T20:24:38.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4640979968 bytes 2026-02-19T20:24:38.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4647820288 bytes 2026-02-19T20:24:38.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4692700160 bytes 2026-02-19T20:24:38.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4737183744 bytes 2026-02-19T20:24:38.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4771678208 bytes 2026-02-19T20:24:38.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:24:38.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:24:38.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:24:38.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:38.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:24:38.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_36 2026-02-19T20:24:38.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_36 snapshot 128M 2026-02-19T20:24:38.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:38.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_36 128M 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:38.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:24:38.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:38.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:24:38.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:24:38.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:24:38.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:38.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_36 2026-02-19T20:24:38.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:24:38.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:38.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:24:38.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:38.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_36 snapshot 2026-02-19T20:24:38.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:38.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:38.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:24:38.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:38.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_36 snapshot 2026-02-19T20:24:39.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:24:39.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_36 2026-02-19T20:24:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_36': 2026-02-19T20:24:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:24:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:24:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:24:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1424f618d251 2026-02-19T20:24:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1424f618d251 2026-02-19T20:24:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:24:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:24:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:38 2026 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:38 2026 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:38 2026 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: a86dfcae-7da8-44fb-8bfe-52f9b7766c69 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_36 2026-02-19T20:24:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:24:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:24:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:24:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:24:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_36 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:24:39.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 14802944 bytes 2026-02-19T20:24:39.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 38504448 bytes 2026-02-19T20:24:39.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 92000256 bytes 2026-02-19T20:24:39.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 99802112 bytes 2026-02-19T20:24:39.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 101524480 bytes 2026-02-19T20:24:39.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 178359296 bytes 2026-02-19T20:24:40.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 224552960 bytes 2026-02-19T20:24:40.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 286321664 bytes 2026-02-19T20:24:40.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 317404160 bytes 2026-02-19T20:24:40.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 365165568 bytes 2026-02-19T20:24:40.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 390659072 bytes 2026-02-19T20:24:40.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 423335936 bytes 2026-02-19T20:24:40.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 447655936 bytes 2026-02-19T20:24:40.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 482406400 bytes 2026-02-19T20:24:40.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 528751616 bytes 2026-02-19T20:24:40.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 593170432 bytes 2026-02-19T20:24:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 641630208 bytes 2026-02-19T20:24:40.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 697052160 bytes 2026-02-19T20:24:40.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 744965120 bytes 2026-02-19T20:24:40.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 797800448 bytes 2026-02-19T20:24:40.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 828615680 bytes 2026-02-19T20:24:40.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 900655104 bytes 2026-02-19T20:24:41.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 935309312 bytes 2026-02-19T20:24:41.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 970108928 bytes 2026-02-19T20:24:41.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1008331776 bytes 2026-02-19T20:24:41.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1029786624 bytes 2026-02-19T20:24:41.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1087196160 bytes 2026-02-19T20:24:41.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1166194688 bytes 2026-02-19T20:24:41.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1235567616 bytes 2026-02-19T20:24:41.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1282521088 bytes 2026-02-19T20:24:41.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1324920832 bytes 2026-02-19T20:24:41.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1369962496 bytes 2026-02-19T20:24:41.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1379057664 bytes 2026-02-19T20:24:41.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1403856896 bytes 2026-02-19T20:24:41.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1444431872 bytes 2026-02-19T20:24:41.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1531771904 bytes 2026-02-19T20:24:41.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1564335104 bytes 2026-02-19T20:24:42.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1586774016 bytes 2026-02-19T20:24:42.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1619537920 bytes 2026-02-19T20:24:42.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1647035392 bytes 2026-02-19T20:24:42.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1692741632 bytes 2026-02-19T20:24:42.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1720194048 bytes 2026-02-19T20:24:42.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1774985216 bytes 2026-02-19T20:24:42.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1805288448 bytes 2026-02-19T20:24:42.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1848196096 bytes 2026-02-19T20:24:42.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1881932800 bytes 2026-02-19T20:24:42.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1972015104 bytes 2026-02-19T20:24:42.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1988534272 bytes 2026-02-19T20:24:42.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2008369152 bytes 2026-02-19T20:24:42.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2055360512 bytes 2026-02-19T20:24:42.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2108465152 bytes 2026-02-19T20:24:42.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2158918656 bytes 2026-02-19T20:24:42.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2166684672 bytes 2026-02-19T20:24:42.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2282934272 bytes 2026-02-19T20:24:43.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2304256000 bytes 2026-02-19T20:24:43.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2359038976 bytes 2026-02-19T20:24:43.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2364072960 bytes 2026-02-19T20:24:43.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2405625856 bytes 2026-02-19T20:24:43.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2464143360 bytes 2026-02-19T20:24:43.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2512929792 bytes 2026-02-19T20:24:43.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2557240320 bytes 2026-02-19T20:24:43.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2617837568 bytes 2026-02-19T20:24:43.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2657147904 bytes 2026-02-19T20:24:43.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2679579648 bytes 2026-02-19T20:24:43.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2680205312 bytes 2026-02-19T20:24:43.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2714477568 bytes 2026-02-19T20:24:43.761 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:24:43.757+0000 7f5bf91cc640 -1 reset not still connected to 0x5be40d0 2026-02-19T20:24:43.761 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:24:43.757+0000 7f5bf91cc640 -1 reset not still connected to 0x10dec4e0 2026-02-19T20:24:43.761 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:24:43.757+0000 7f5bf91cc640 -1 reset not still connected to 0x12839ee0 2026-02-19T20:24:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2765308928 bytes 2026-02-19T20:24:43.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2788680704 bytes 2026-02-19T20:24:43.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2817637376 bytes 2026-02-19T20:24:43.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2883188736 bytes 2026-02-19T20:24:44.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2937048064 bytes 2026-02-19T20:24:44.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2953837568 bytes 2026-02-19T20:24:44.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2980085760 bytes 2026-02-19T20:24:44.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3028925440 bytes 2026-02-19T20:24:44.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3087719424 bytes 2026-02-19T20:24:44.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3108921344 bytes 2026-02-19T20:24:44.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3135127552 bytes 2026-02-19T20:24:44.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3173954560 bytes 2026-02-19T20:24:44.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3257184256 bytes 2026-02-19T20:24:44.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3320370176 bytes 2026-02-19T20:24:44.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3404140544 bytes 2026-02-19T20:24:44.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3419104256 bytes 2026-02-19T20:24:44.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3465721856 bytes 2026-02-19T20:24:44.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3506340864 bytes 2026-02-19T20:24:44.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3579839488 bytes 2026-02-19T20:24:45.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3645473792 bytes 2026-02-19T20:24:45.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3694561280 bytes 2026-02-19T20:24:45.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3799916544 bytes 2026-02-19T20:24:45.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3825171456 bytes 2026-02-19T20:24:45.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3831472128 bytes 2026-02-19T20:24:45.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3858544640 bytes 2026-02-19T20:24:45.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3872800768 bytes 2026-02-19T20:24:45.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3886910464 bytes 2026-02-19T20:24:45.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3910046720 bytes 2026-02-19T20:24:45.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3958492160 bytes 2026-02-19T20:24:45.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4029041664 bytes 2026-02-19T20:24:45.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4029664256 bytes 2026-02-19T20:24:45.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4072523776 bytes 2026-02-19T20:24:45.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4089829376 bytes 2026-02-19T20:24:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4134369280 bytes 2026-02-19T20:24:45.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4204366848 bytes 2026-02-19T20:24:46.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4288704512 bytes 2026-02-19T20:24:46.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4349551616 bytes 2026-02-19T20:24:46.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4373988352 bytes 2026-02-19T20:24:46.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4447781888 bytes 2026-02-19T20:24:46.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4494264320 bytes 2026-02-19T20:24:46.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4539193344 bytes 2026-02-19T20:24:46.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4572753920 bytes 2026-02-19T20:24:46.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4579144704 bytes 2026-02-19T20:24:46.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4618162176 bytes 2026-02-19T20:24:46.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4719280128 bytes 2026-02-19T20:24:46.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4741666816 bytes 2026-02-19T20:24:46.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4816028672 bytes 2026-02-19T20:24:46.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4892157952 bytes 2026-02-19T20:24:46.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4908212224 bytes 2026-02-19T20:24:46.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4958861312 bytes 2026-02-19T20:24:47.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4998748160 bytes 2026-02-19T20:24:47.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5049391104 bytes 2026-02-19T20:24:47.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5079502848 bytes 2026-02-19T20:24:47.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5108420608 bytes 2026-02-19T20:24:47.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5188089856 bytes 2026-02-19T20:24:47.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5228345344 bytes 2026-02-19T20:24:47.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5263897600 bytes 2026-02-19T20:24:47.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5274954752 bytes 2026-02-19T20:24:47.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5372518400 bytes 2026-02-19T20:24:47.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5409528832 bytes 2026-02-19T20:24:47.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5413444608 bytes 2026-02-19T20:24:47.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5432467456 bytes 2026-02-19T20:24:47.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5470381056 bytes 2026-02-19T20:24:47.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5503551488 bytes 2026-02-19T20:24:47.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5540199424 bytes 2026-02-19T20:24:47.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5565271040 bytes 2026-02-19T20:24:47.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5603920896 bytes 2026-02-19T20:24:47.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5627020288 bytes 2026-02-19T20:24:48.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5664404480 bytes 2026-02-19T20:24:48.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5666487296 bytes 2026-02-19T20:24:48.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5712410624 bytes 2026-02-19T20:24:48.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5777315840 bytes 2026-02-19T20:24:48.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5802687488 bytes 2026-02-19T20:24:48.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5816792064 bytes 2026-02-19T20:24:48.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5891860480 bytes 2026-02-19T20:24:48.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5960688640 bytes 2026-02-19T20:24:48.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5988282368 bytes 2026-02-19T20:24:48.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6032769024 bytes 2026-02-19T20:24:48.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6081815552 bytes 2026-02-19T20:24:48.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6161705984 bytes 2026-02-19T20:24:48.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6259516416 bytes 2026-02-19T20:24:48.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6311692288 bytes 2026-02-19T20:24:48.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6366052352 bytes 2026-02-19T20:24:49.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6435615744 bytes 2026-02-19T20:24:49.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6436214784 bytes 2026-02-19T20:24:49.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6463749120 bytes 2026-02-19T20:24:49.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6503752704 bytes 2026-02-19T20:24:49.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6525235200 bytes 2026-02-19T20:24:49.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6558433280 bytes 2026-02-19T20:24:49.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6605672448 bytes 2026-02-19T20:24:49.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6650715136 bytes 2026-02-19T20:24:49.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6660387840 bytes 2026-02-19T20:24:49.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6716035072 bytes 2026-02-19T20:24:49.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:24:49.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_37 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_37 snapshot 128M 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:49.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_37 128M 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:49.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:24:49.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:49.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:24:49.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:24:49.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:24:49.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:24:49.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_37 2026-02-19T20:24:49.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:24:49.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:49.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:24:49.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:24:49.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_37 snapshot 2026-02-19T20:24:49.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:49.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:49.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:24:49.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:24:49.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_37 snapshot 2026-02-19T20:24:50.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:24:50.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_37 2026-02-19T20:24:50.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_37': 2026-02-19T20:24:50.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1430968c969c 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1430968c969c 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:49 2026 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:49 2026 2026-02-19T20:24:50.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:49 2026 2026-02-19T20:24:50.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:24:50.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:24:50.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: a1238f9d-e6e7-407f-88bc-b2822ea2d0df 2026-02-19T20:24:50.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:24:50.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:24:50.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_37 2026-02-19T20:24:50.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:24:50.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:24:50.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:24:50.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:24:50.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:24:50.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:24:50.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_37 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:24:50.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 33328128 bytes 2026-02-19T20:24:50.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 59721728 bytes 2026-02-19T20:24:50.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 76833792 bytes 2026-02-19T20:24:51.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 141986816 bytes 2026-02-19T20:24:51.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 146902016 bytes 2026-02-19T20:24:51.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 202357760 bytes 2026-02-19T20:24:51.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 215147520 bytes 2026-02-19T20:24:51.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 285777920 bytes 2026-02-19T20:24:51.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 314247168 bytes 2026-02-19T20:24:51.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 381663232 bytes 2026-02-19T20:24:51.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 414461952 bytes 2026-02-19T20:24:51.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 438256640 bytes 2026-02-19T20:24:51.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 533603328 bytes 2026-02-19T20:24:51.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 565074944 bytes 2026-02-19T20:24:51.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 593889280 bytes 2026-02-19T20:24:51.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 628376576 bytes 2026-02-19T20:24:51.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 667992064 bytes 2026-02-19T20:24:51.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 716854272 bytes 2026-02-19T20:24:51.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 754008064 bytes 2026-02-19T20:24:52.049 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 824506368 bytes 2026-02-19T20:24:52.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 831707136 bytes 2026-02-19T20:24:52.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 903073792 bytes 2026-02-19T20:24:52.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 920294400 bytes 2026-02-19T20:24:52.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 962267136 bytes 2026-02-19T20:24:52.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1048216576 bytes 2026-02-19T20:24:52.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1090665472 bytes 2026-02-19T20:24:52.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1115558912 bytes 2026-02-19T20:24:52.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1174440960 bytes 2026-02-19T20:24:52.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1178427392 bytes 2026-02-19T20:24:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1231044608 bytes 2026-02-19T20:24:52.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1245792256 bytes 2026-02-19T20:24:52.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1284917248 bytes 2026-02-19T20:24:52.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1307853824 bytes 2026-02-19T20:24:52.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1347840000 bytes 2026-02-19T20:24:52.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1357739008 bytes 2026-02-19T20:24:52.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1373081600 bytes 2026-02-19T20:24:52.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1395243008 bytes 2026-02-19T20:24:53.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1395853312 bytes 2026-02-19T20:24:53.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1450606592 bytes 2026-02-19T20:24:53.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1519256576 bytes 2026-02-19T20:24:53.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1560811520 bytes 2026-02-19T20:24:53.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1670508544 bytes 2026-02-19T20:24:53.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1687586816 bytes 2026-02-19T20:24:53.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1735081984 bytes 2026-02-19T20:24:53.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1766402048 bytes 2026-02-19T20:24:53.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1825821696 bytes 2026-02-19T20:24:53.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1902116864 bytes 2026-02-19T20:24:53.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1933808640 bytes 2026-02-19T20:24:53.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1965974528 bytes 2026-02-19T20:24:53.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2021915648 bytes 2026-02-19T20:24:53.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2058836992 bytes 2026-02-19T20:24:53.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2107189248 bytes 2026-02-19T20:24:54.056 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2141038592 bytes 2026-02-19T20:24:54.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2198425600 bytes 2026-02-19T20:24:54.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2236254208 bytes 2026-02-19T20:24:54.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2294036480 bytes 2026-02-19T20:24:54.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2338323456 bytes 2026-02-19T20:24:54.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2338952192 bytes 2026-02-19T20:24:54.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2353819648 bytes 2026-02-19T20:24:54.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2448141312 bytes 2026-02-19T20:24:54.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2470733824 bytes 2026-02-19T20:24:54.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2498991104 bytes 2026-02-19T20:24:54.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2502408192 bytes 2026-02-19T20:24:54.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2576643072 bytes 2026-02-19T20:24:54.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2626222080 bytes 2026-02-19T20:24:54.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2657553408 bytes 2026-02-19T20:24:54.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2667261952 bytes 2026-02-19T20:24:54.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2696799232 bytes 2026-02-19T20:24:54.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2752743424 bytes 2026-02-19T20:24:55.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2767604736 bytes 2026-02-19T20:24:55.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2768218112 bytes 2026-02-19T20:24:55.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2820002816 bytes 2026-02-19T20:24:55.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2852444160 bytes 2026-02-19T20:24:55.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2891412480 bytes 2026-02-19T20:24:55.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2925523968 bytes 2026-02-19T20:24:55.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 2937762816 bytes 2026-02-19T20:24:55.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 2985929728 bytes 2026-02-19T20:24:55.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3014492160 bytes 2026-02-19T20:24:55.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3082537984 bytes 2026-02-19T20:24:55.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3127005184 bytes 2026-02-19T20:24:55.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3133541376 bytes 2026-02-19T20:24:55.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3170506752 bytes 2026-02-19T20:24:55.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3212837888 bytes 2026-02-19T20:24:55.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3228653568 bytes 2026-02-19T20:24:55.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3240532992 bytes 2026-02-19T20:24:55.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3275614208 bytes 2026-02-19T20:24:55.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3290707968 bytes 2026-02-19T20:24:55.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3295856640 bytes 2026-02-19T20:24:56.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3349306368 bytes 2026-02-19T20:24:56.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3371084800 bytes 2026-02-19T20:24:56.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3389036544 bytes 2026-02-19T20:24:56.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3420838912 bytes 2026-02-19T20:24:56.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3463578624 bytes 2026-02-19T20:24:56.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3467689984 bytes 2026-02-19T20:24:56.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3486214144 bytes 2026-02-19T20:24:56.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3500596224 bytes 2026-02-19T20:24:56.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3587968000 bytes 2026-02-19T20:24:56.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3655910400 bytes 2026-02-19T20:24:56.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3661768704 bytes 2026-02-19T20:24:56.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 3693106176 bytes 2026-02-19T20:24:56.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 3743197184 bytes 2026-02-19T20:24:56.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 3748775936 bytes 2026-02-19T20:24:56.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 3780855808 bytes 2026-02-19T20:24:56.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 3818853376 bytes 2026-02-19T20:24:56.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 3838411776 bytes 2026-02-19T20:24:56.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 3863778304 bytes 2026-02-19T20:24:57.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 3907471360 bytes 2026-02-19T20:24:57.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 3957794816 bytes 2026-02-19T20:24:57.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4011906048 bytes 2026-02-19T20:24:57.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4081240064 bytes 2026-02-19T20:24:57.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4233959424 bytes 2026-02-19T20:24:57.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4255266816 bytes 2026-02-19T20:24:57.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4285142016 bytes 2026-02-19T20:24:57.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4322763776 bytes 2026-02-19T20:24:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4392757248 bytes 2026-02-19T20:24:57.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4393378816 bytes 2026-02-19T20:24:57.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4427932672 bytes 2026-02-19T20:24:57.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4452209664 bytes 2026-02-19T20:24:57.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4488643584 bytes 2026-02-19T20:24:57.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4499273728 bytes 2026-02-19T20:24:57.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4523745280 bytes 2026-02-19T20:24:57.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4554816512 bytes 2026-02-19T20:24:57.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4609938432 bytes 2026-02-19T20:24:58.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4678994944 bytes 2026-02-19T20:24:58.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 4753240064 bytes 2026-02-19T20:24:58.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 4784338944 bytes 2026-02-19T20:24:58.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 4814391296 bytes 2026-02-19T20:24:58.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 4842878976 bytes 2026-02-19T20:24:58.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 4868467712 bytes 2026-02-19T20:24:58.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 4870479872 bytes 2026-02-19T20:24:58.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 4895559680 bytes 2026-02-19T20:24:58.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 4949876736 bytes 2026-02-19T20:24:58.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5004639232 bytes 2026-02-19T20:24:58.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5024418816 bytes 2026-02-19T20:24:58.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5049679872 bytes 2026-02-19T20:24:58.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5086075904 bytes 2026-02-19T20:24:58.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5128769536 bytes 2026-02-19T20:24:58.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5139946496 bytes 2026-02-19T20:24:58.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5161732096 bytes 2026-02-19T20:24:58.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5216567296 bytes 2026-02-19T20:24:59.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5239073792 bytes 2026-02-19T20:24:59.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5244910592 bytes 2026-02-19T20:24:59.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5282874368 bytes 2026-02-19T20:24:59.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5291213824 bytes 2026-02-19T20:24:59.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5328402432 bytes 2026-02-19T20:24:59.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5373145088 bytes 2026-02-19T20:24:59.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5394726912 bytes 2026-02-19T20:24:59.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5443014656 bytes 2026-02-19T20:24:59.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 5495543808 bytes 2026-02-19T20:24:59.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 5512795136 bytes 2026-02-19T20:24:59.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 5571516416 bytes 2026-02-19T20:24:59.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 5603453952 bytes 2026-02-19T20:24:59.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 5661627392 bytes 2026-02-19T20:24:59.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 5662226432 bytes 2026-02-19T20:24:59.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 5725166592 bytes 2026-02-19T20:24:59.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 5769963520 bytes 2026-02-19T20:24:59.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 5797504000 bytes 2026-02-19T20:24:59.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 5877078016 bytes 2026-02-19T20:25:00.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 5901616128 bytes 2026-02-19T20:25:00.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 5949678592 bytes 2026-02-19T20:25:00.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 5984146432 bytes 2026-02-19T20:25:00.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6040469504 bytes 2026-02-19T20:25:00.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6121672704 bytes 2026-02-19T20:25:00.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6190941184 bytes 2026-02-19T20:25:00.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6223537152 bytes 2026-02-19T20:25:00.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6268405760 bytes 2026-02-19T20:25:00.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6309007360 bytes 2026-02-19T20:25:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6340718592 bytes 2026-02-19T20:25:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6362836992 bytes 2026-02-19T20:25:00.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6386587648 bytes 2026-02-19T20:25:00.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6392632320 bytes 2026-02-19T20:25:00.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 6473602048 bytes 2026-02-19T20:25:00.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 6512956416 bytes 2026-02-19T20:25:01.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 6538580992 bytes 2026-02-19T20:25:01.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 6648231936 bytes 2026-02-19T20:25:01.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 6666288128 bytes 2026-02-19T20:25:01.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 6712207360 bytes 2026-02-19T20:25:01.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 6775592960 bytes 2026-02-19T20:25:01.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 6809322496 bytes 2026-02-19T20:25:01.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 6879852544 bytes 2026-02-19T20:25:01.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 6954435584 bytes 2026-02-19T20:25:01.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 6972710912 bytes 2026-02-19T20:25:01.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 6994950144 bytes 2026-02-19T20:25:01.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7075365888 bytes 2026-02-19T20:25:01.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7170352128 bytes 2026-02-19T20:25:01.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7249045504 bytes 2026-02-19T20:25:01.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7305701376 bytes 2026-02-19T20:25:02.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7366781952 bytes 2026-02-19T20:25:02.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7384269824 bytes 2026-02-19T20:25:02.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7452312576 bytes 2026-02-19T20:25:02.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7504962560 bytes 2026-02-19T20:25:02.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7514343424 bytes 2026-02-19T20:25:02.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7632624640 bytes 2026-02-19T20:25:02.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 7684939776 bytes 2026-02-19T20:25:02.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 7755182080 bytes 2026-02-19T20:25:02.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:25:02.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_38 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_38 snapshot 128M 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:25:02.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_38 128M 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:25:02.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:25:02.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:25:02.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:02.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_38 2026-02-19T20:25:02.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:25:02.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:02.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:25:02.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:02.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_38 snapshot 2026-02-19T20:25:02.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:02.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:02.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:25:02.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:02.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_38 snapshot 2026-02-19T20:25:03.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:25:03.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_38 2026-02-19T20:25:03.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_38': 2026-02-19T20:25:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:25:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:25:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:25:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 143cdd2befaa 2026-02-19T20:25:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.143cdd2befaa 2026-02-19T20:25:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:25:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:25:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:25:03.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:25:03.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:02 2026 2026-02-19T20:25:03.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:02 2026 2026-02-19T20:25:03.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:02 2026 2026-02-19T20:25:03.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:25:03.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:25:03.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: bfe65774-7a8a-43f3-a6a6-84525bb3df1b 2026-02-19T20:25:03.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:25:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:25:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_38 2026-02-19T20:25:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:25:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:25:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=11 2026-02-19T20:25:03.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:25:03.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_38 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:25:03.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 61939712 bytes 2026-02-19T20:25:03.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 86552576 bytes 2026-02-19T20:25:03.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 174613504 bytes 2026-02-19T20:25:03.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 234802176 bytes 2026-02-19T20:25:03.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 276115456 bytes 2026-02-19T20:25:04.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 280401920 bytes 2026-02-19T20:25:04.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 369790976 bytes 2026-02-19T20:25:04.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 417336320 bytes 2026-02-19T20:25:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 461655040 bytes 2026-02-19T20:25:04.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 559020032 bytes 2026-02-19T20:25:04.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 626751488 bytes 2026-02-19T20:25:04.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 704714752 bytes 2026-02-19T20:25:04.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 738654208 bytes 2026-02-19T20:25:04.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 759058432 bytes 2026-02-19T20:25:04.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 772645888 bytes 2026-02-19T20:25:04.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 850963456 bytes 2026-02-19T20:25:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 868658176 bytes 2026-02-19T20:25:04.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 884952064 bytes 2026-02-19T20:25:04.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 941217792 bytes 2026-02-19T20:25:04.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 963471360 bytes 2026-02-19T20:25:05.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1013996544 bytes 2026-02-19T20:25:05.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1064131584 bytes 2026-02-19T20:25:05.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1071365120 bytes 2026-02-19T20:25:05.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1109560320 bytes 2026-02-19T20:25:05.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1172217856 bytes 2026-02-19T20:25:05.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1213676544 bytes 2026-02-19T20:25:05.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1264382976 bytes 2026-02-19T20:25:05.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1314387968 bytes 2026-02-19T20:25:05.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1375032320 bytes 2026-02-19T20:25:05.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1383905280 bytes 2026-02-19T20:25:05.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1411454976 bytes 2026-02-19T20:25:05.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1445815296 bytes 2026-02-19T20:25:05.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1482310656 bytes 2026-02-19T20:25:05.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1560070144 bytes 2026-02-19T20:25:05.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1589575680 bytes 2026-02-19T20:25:05.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1610718208 bytes 2026-02-19T20:25:05.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1627035648 bytes 2026-02-19T20:25:06.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1634952192 bytes 2026-02-19T20:25:06.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1711077376 bytes 2026-02-19T20:25:06.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1779650560 bytes 2026-02-19T20:25:06.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1830385664 bytes 2026-02-19T20:25:06.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1948336128 bytes 2026-02-19T20:25:06.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1996479488 bytes 2026-02-19T20:25:06.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2073896960 bytes 2026-02-19T20:25:06.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2094411776 bytes 2026-02-19T20:25:06.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2108793856 bytes 2026-02-19T20:25:06.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2154867712 bytes 2026-02-19T20:25:06.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2190727168 bytes 2026-02-19T20:25:06.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2254937088 bytes 2026-02-19T20:25:06.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2296002560 bytes 2026-02-19T20:25:06.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2384139264 bytes 2026-02-19T20:25:06.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2384747520 bytes 2026-02-19T20:25:07.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2400421888 bytes 2026-02-19T20:25:07.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2462669824 bytes 2026-02-19T20:25:07.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2468489216 bytes 2026-02-19T20:25:07.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2517057536 bytes 2026-02-19T20:25:07.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2576297984 bytes 2026-02-19T20:25:07.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2673278976 bytes 2026-02-19T20:25:07.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2751316992 bytes 2026-02-19T20:25:07.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2756031488 bytes 2026-02-19T20:25:07.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2792724480 bytes 2026-02-19T20:25:07.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2907585536 bytes 2026-02-19T20:25:07.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2949740544 bytes 2026-02-19T20:25:07.704 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.39467605202508926) > weight (0.25) 2026-02-19T20:25:07.704 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2026-02-19T20:25:07.705 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2026-02-19T20:25:07.705 INFO:tasks.rbd_mirror_thrash:waiting for 7.073986910593245 secs before reviving daemons 2026-02-19T20:25:07.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:25:07.702+0000 7f6c78fe9640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 48042) UID: 1000 2026-02-19T20:25:07.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2996091904 bytes 2026-02-19T20:25:07.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3021076480 bytes 2026-02-19T20:25:07.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3052320768 bytes 2026-02-19T20:25:07.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3107047424 bytes 2026-02-19T20:25:07.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3174369280 bytes 2026-02-19T20:25:08.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3190761472 bytes 2026-02-19T20:25:08.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3220526080 bytes 2026-02-19T20:25:08.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3283635200 bytes 2026-02-19T20:25:08.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3300675584 bytes 2026-02-19T20:25:08.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3366183936 bytes 2026-02-19T20:25:08.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3429513216 bytes 2026-02-19T20:25:08.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3456965632 bytes 2026-02-19T20:25:08.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3502678016 bytes 2026-02-19T20:25:08.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3507227648 bytes 2026-02-19T20:25:08.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3513951232 bytes 2026-02-19T20:25:08.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3541227520 bytes 2026-02-19T20:25:08.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3583688704 bytes 2026-02-19T20:25:08.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3636960256 bytes 2026-02-19T20:25:08.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3663424512 bytes 2026-02-19T20:25:08.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3707437056 bytes 2026-02-19T20:25:08.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3745511424 bytes 2026-02-19T20:25:08.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3773510656 bytes 2026-02-19T20:25:08.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3820931072 bytes 2026-02-19T20:25:09.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3824992256 bytes 2026-02-19T20:25:09.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3840792576 bytes 2026-02-19T20:25:09.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3889386496 bytes 2026-02-19T20:25:09.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3937175552 bytes 2026-02-19T20:25:09.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4023395328 bytes 2026-02-19T20:25:09.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4047376384 bytes 2026-02-19T20:25:09.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4083461120 bytes 2026-02-19T20:25:09.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4098557952 bytes 2026-02-19T20:25:09.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4136668160 bytes 2026-02-19T20:25:09.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4140299264 bytes 2026-02-19T20:25:09.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4152858624 bytes 2026-02-19T20:25:09.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4220400640 bytes 2026-02-19T20:25:09.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4256817152 bytes 2026-02-19T20:25:09.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4305029120 bytes 2026-02-19T20:25:10.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4338384896 bytes 2026-02-19T20:25:10.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4380769280 bytes 2026-02-19T20:25:10.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4437322752 bytes 2026-02-19T20:25:10.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4449150976 bytes 2026-02-19T20:25:10.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4495888384 bytes 2026-02-19T20:25:10.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4537023488 bytes 2026-02-19T20:25:10.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4592052224 bytes 2026-02-19T20:25:10.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4655544320 bytes 2026-02-19T20:25:10.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4663000064 bytes 2026-02-19T20:25:10.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4745677824 bytes 2026-02-19T20:25:10.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4802036736 bytes 2026-02-19T20:25:10.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4821502976 bytes 2026-02-19T20:25:10.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4854567936 bytes 2026-02-19T20:25:10.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4879179776 bytes 2026-02-19T20:25:10.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4941647872 bytes 2026-02-19T20:25:10.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4969103360 bytes 2026-02-19T20:25:11.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5045547008 bytes 2026-02-19T20:25:11.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5049821184 bytes 2026-02-19T20:25:11.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5140581376 bytes 2026-02-19T20:25:11.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5212102656 bytes 2026-02-19T20:25:11.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5243344896 bytes 2026-02-19T20:25:11.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5310621696 bytes 2026-02-19T20:25:11.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5334679552 bytes 2026-02-19T20:25:11.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5405506560 bytes 2026-02-19T20:25:11.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5459639296 bytes 2026-02-19T20:25:11.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5491201024 bytes 2026-02-19T20:25:11.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5590563840 bytes 2026-02-19T20:25:11.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5640805376 bytes 2026-02-19T20:25:11.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5725803520 bytes 2026-02-19T20:25:11.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5753757696 bytes 2026-02-19T20:25:11.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5816076288 bytes 2026-02-19T20:25:12.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5831575552 bytes 2026-02-19T20:25:12.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5878683648 bytes 2026-02-19T20:25:12.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5890008064 bytes 2026-02-19T20:25:12.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5917264896 bytes 2026-02-19T20:25:12.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5959866368 bytes 2026-02-19T20:25:12.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5982703616 bytes 2026-02-19T20:25:12.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6006759424 bytes 2026-02-19T20:25:12.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6062786560 bytes 2026-02-19T20:25:12.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6074522624 bytes 2026-02-19T20:25:12.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6138554368 bytes 2026-02-19T20:25:12.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6157548544 bytes 2026-02-19T20:25:12.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6161189888 bytes 2026-02-19T20:25:12.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6205860864 bytes 2026-02-19T20:25:12.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6283005952 bytes 2026-02-19T20:25:12.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6343847936 bytes 2026-02-19T20:25:12.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6375028736 bytes 2026-02-19T20:25:12.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6442341376 bytes 2026-02-19T20:25:12.935 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6477437952 bytes 2026-02-19T20:25:12.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6496161792 bytes 2026-02-19T20:25:13.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6527903744 bytes 2026-02-19T20:25:13.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6565646336 bytes 2026-02-19T20:25:13.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6620665856 bytes 2026-02-19T20:25:13.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6633141248 bytes 2026-02-19T20:25:13.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6663734272 bytes 2026-02-19T20:25:13.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6706950144 bytes 2026-02-19T20:25:13.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6793338880 bytes 2026-02-19T20:25:13.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6834762752 bytes 2026-02-19T20:25:13.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6847002624 bytes 2026-02-19T20:25:13.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6874580992 bytes 2026-02-19T20:25:13.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6899856384 bytes 2026-02-19T20:25:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6970206208 bytes 2026-02-19T20:25:13.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7045402624 bytes 2026-02-19T20:25:13.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7065839616 bytes 2026-02-19T20:25:13.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7094039552 bytes 2026-02-19T20:25:13.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7119102976 bytes 2026-02-19T20:25:13.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7178816512 bytes 2026-02-19T20:25:14.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7220299776 bytes 2026-02-19T20:25:14.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7238596608 bytes 2026-02-19T20:25:14.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7240832000 bytes 2026-02-19T20:25:14.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7313970176 bytes 2026-02-19T20:25:14.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7348644864 bytes 2026-02-19T20:25:14.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7349263360 bytes 2026-02-19T20:25:14.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7388841984 bytes 2026-02-19T20:25:14.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7421993984 bytes 2026-02-19T20:25:14.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7434589184 bytes 2026-02-19T20:25:14.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7472352256 bytes 2026-02-19T20:25:14.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7496732672 bytes 2026-02-19T20:25:14.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7502909440 bytes 2026-02-19T20:25:14.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7524678656 bytes 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_39 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_39 snapshot 128M 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_39 128M 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:14.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_39 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_39 snapshot 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:14.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_39 snapshot 2026-02-19T20:25:14.779 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2026-02-19T20:25:14.779 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:25:14.780 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2026-02-19T20:25:14.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2026-02-19T20:25:14.780 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2026-02-19T20:25:14.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2026-02-19T20:25:14.783 INFO:tasks.rbd_mirror_thrash:waiting for 88.93650764100444 secs before thrashing 2026-02-19T20:25:15.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:25:15.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_39 2026-02-19T20:25:15.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_39': 2026-02-19T20:25:15.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1448231763cc 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1448231763cc 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:14 2026 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:14 2026 2026-02-19T20:25:15.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:14 2026 2026-02-19T20:25:15.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:25:15.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:25:15.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 170be547-d7a9-46a7-b6db-d32c6a98434d 2026-02-19T20:25:15.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:25:15.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:25:15.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_39 2026-02-19T20:25:15.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:15.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:15.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:25:15.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:25:15.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:25:15.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:25:15.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_39 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:25:15.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 635904 bytes 2026-02-19T20:25:15.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 20793344 bytes 2026-02-19T20:25:16.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 142611456 bytes 2026-02-19T20:25:16.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 249129984 bytes 2026-02-19T20:25:16.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 266314752 bytes 2026-02-19T20:25:16.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 298404864 bytes 2026-02-19T20:25:16.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 387055616 bytes 2026-02-19T20:25:16.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 440123392 bytes 2026-02-19T20:25:16.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 444565504 bytes 2026-02-19T20:25:16.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 466091008 bytes 2026-02-19T20:25:16.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 466709504 bytes 2026-02-19T20:25:16.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 517998592 bytes 2026-02-19T20:25:16.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 552644608 bytes 2026-02-19T20:25:16.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 572813312 bytes 2026-02-19T20:25:16.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 614684672 bytes 2026-02-19T20:25:16.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 647988224 bytes 2026-02-19T20:25:16.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 713221120 bytes 2026-02-19T20:25:16.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 735095808 bytes 2026-02-19T20:25:16.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 757846016 bytes 2026-02-19T20:25:17.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 865374208 bytes 2026-02-19T20:25:17.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 891224064 bytes 2026-02-19T20:25:17.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 931004416 bytes 2026-02-19T20:25:17.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 952551424 bytes 2026-02-19T20:25:17.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1007101952 bytes 2026-02-19T20:25:17.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1047824384 bytes 2026-02-19T20:25:17.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1093974016 bytes 2026-02-19T20:25:17.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1115279360 bytes 2026-02-19T20:25:17.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1126985728 bytes 2026-02-19T20:25:17.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1183223808 bytes 2026-02-19T20:25:17.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1191467008 bytes 2026-02-19T20:25:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1248303104 bytes 2026-02-19T20:25:17.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1302501376 bytes 2026-02-19T20:25:17.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1322701824 bytes 2026-02-19T20:25:17.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1332160512 bytes 2026-02-19T20:25:17.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1360507904 bytes 2026-02-19T20:25:17.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1426952192 bytes 2026-02-19T20:25:17.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1442053120 bytes 2026-02-19T20:25:18.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1466183680 bytes 2026-02-19T20:25:18.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1500329984 bytes 2026-02-19T20:25:18.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1543304192 bytes 2026-02-19T20:25:18.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1597856768 bytes 2026-02-19T20:25:18.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1623026688 bytes 2026-02-19T20:25:18.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1657313280 bytes 2026-02-19T20:25:18.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1682318336 bytes 2026-02-19T20:25:18.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1747790848 bytes 2026-02-19T20:25:18.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1808686080 bytes 2026-02-19T20:25:18.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1848588288 bytes 2026-02-19T20:25:18.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1882151936 bytes 2026-02-19T20:25:18.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1947927552 bytes 2026-02-19T20:25:18.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1970212864 bytes 2026-02-19T20:25:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2045672448 bytes 2026-02-19T20:25:18.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2108397568 bytes 2026-02-19T20:25:18.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2161534976 bytes 2026-02-19T20:25:19.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2263680000 bytes 2026-02-19T20:25:19.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2292210688 bytes 2026-02-19T20:25:19.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2338498560 bytes 2026-02-19T20:25:19.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2410898432 bytes 2026-02-19T20:25:19.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2473318400 bytes 2026-02-19T20:25:19.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2527715328 bytes 2026-02-19T20:25:19.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2567261184 bytes 2026-02-19T20:25:19.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2628747264 bytes 2026-02-19T20:25:19.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2645568512 bytes 2026-02-19T20:25:19.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2670030848 bytes 2026-02-19T20:25:19.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2686083072 bytes 2026-02-19T20:25:19.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2753696768 bytes 2026-02-19T20:25:19.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2781834240 bytes 2026-02-19T20:25:19.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2805424128 bytes 2026-02-19T20:25:19.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2826839040 bytes 2026-02-19T20:25:19.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2890013696 bytes 2026-02-19T20:25:20.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2963375104 bytes 2026-02-19T20:25:20.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2976315392 bytes 2026-02-19T20:25:20.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3062431744 bytes 2026-02-19T20:25:20.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3093001216 bytes 2026-02-19T20:25:20.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3131707392 bytes 2026-02-19T20:25:20.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3163935744 bytes 2026-02-19T20:25:20.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3197973504 bytes 2026-02-19T20:25:20.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3267257344 bytes 2026-02-19T20:25:20.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3370347520 bytes 2026-02-19T20:25:20.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3370971136 bytes 2026-02-19T20:25:20.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3426402304 bytes 2026-02-19T20:25:20.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3472589824 bytes 2026-02-19T20:25:20.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3498784768 bytes 2026-02-19T20:25:20.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3503486976 bytes 2026-02-19T20:25:20.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3513185280 bytes 2026-02-19T20:25:20.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3572244480 bytes 2026-02-19T20:25:20.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3572850688 bytes 2026-02-19T20:25:21.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3591984128 bytes 2026-02-19T20:25:21.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3645496320 bytes 2026-02-19T20:25:21.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3687860224 bytes 2026-02-19T20:25:21.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3733098496 bytes 2026-02-19T20:25:21.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3795824640 bytes 2026-02-19T20:25:21.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3840096256 bytes 2026-02-19T20:25:21.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3896162304 bytes 2026-02-19T20:25:21.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3898081280 bytes 2026-02-19T20:25:21.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3928168448 bytes 2026-02-19T20:25:21.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3961541632 bytes 2026-02-19T20:25:21.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4008993792 bytes 2026-02-19T20:25:21.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4059431936 bytes 2026-02-19T20:25:21.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4172697600 bytes 2026-02-19T20:25:21.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4253708288 bytes 2026-02-19T20:25:21.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4310529024 bytes 2026-02-19T20:25:22.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4367043584 bytes 2026-02-19T20:25:22.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4390730752 bytes 2026-02-19T20:25:22.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4406658048 bytes 2026-02-19T20:25:22.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4426863616 bytes 2026-02-19T20:25:22.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4427456512 bytes 2026-02-19T20:25:22.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4429801472 bytes 2026-02-19T20:25:22.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4463744000 bytes 2026-02-19T20:25:22.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4504804352 bytes 2026-02-19T20:25:22.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4560291840 bytes 2026-02-19T20:25:22.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4591304704 bytes 2026-02-19T20:25:22.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4634118144 bytes 2026-02-19T20:25:22.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4705408000 bytes 2026-02-19T20:25:22.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4857990144 bytes 2026-02-19T20:25:22.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4879212544 bytes 2026-02-19T20:25:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4891714560 bytes 2026-02-19T20:25:22.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4911344640 bytes 2026-02-19T20:25:23.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4985937920 bytes 2026-02-19T20:25:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5004836864 bytes 2026-02-19T20:25:23.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5024352256 bytes 2026-02-19T20:25:23.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5063252992 bytes 2026-02-19T20:25:23.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5076825088 bytes 2026-02-19T20:25:23.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5097080832 bytes 2026-02-19T20:25:23.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5143366656 bytes 2026-02-19T20:25:23.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5207203840 bytes 2026-02-19T20:25:23.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5260697600 bytes 2026-02-19T20:25:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5280956416 bytes 2026-02-19T20:25:23.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5315479552 bytes 2026-02-19T20:25:23.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5358317568 bytes 2026-02-19T20:25:23.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5379125248 bytes 2026-02-19T20:25:23.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5397336064 bytes 2026-02-19T20:25:23.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5399685120 bytes 2026-02-19T20:25:23.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5442470912 bytes 2026-02-19T20:25:23.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5466988544 bytes 2026-02-19T20:25:23.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5467593728 bytes 2026-02-19T20:25:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5501639680 bytes 2026-02-19T20:25:24.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5592871936 bytes 2026-02-19T20:25:24.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5672332288 bytes 2026-02-19T20:25:24.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5688337408 bytes 2026-02-19T20:25:24.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5731489792 bytes 2026-02-19T20:25:24.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5762412544 bytes 2026-02-19T20:25:24.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5788952576 bytes 2026-02-19T20:25:24.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5833034752 bytes 2026-02-19T20:25:24.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5896284160 bytes 2026-02-19T20:25:24.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5906484224 bytes 2026-02-19T20:25:24.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5951261696 bytes 2026-02-19T20:25:24.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5970265088 bytes 2026-02-19T20:25:24.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5998826496 bytes 2026-02-19T20:25:24.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6012874752 bytes 2026-02-19T20:25:24.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6052876288 bytes 2026-02-19T20:25:24.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6084578304 bytes 2026-02-19T20:25:25.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6134680576 bytes 2026-02-19T20:25:25.196 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6169035776 bytes 2026-02-19T20:25:25.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6186781696 bytes 2026-02-19T20:25:25.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6252588032 bytes 2026-02-19T20:25:25.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6266943488 bytes 2026-02-19T20:25:25.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6302102528 bytes 2026-02-19T20:25:25.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6302709760 bytes 2026-02-19T20:25:25.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6367301632 bytes 2026-02-19T20:25:25.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6385992704 bytes 2026-02-19T20:25:25.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6463958016 bytes 2026-02-19T20:25:25.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6479633408 bytes 2026-02-19T20:25:25.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6528635904 bytes 2026-02-19T20:25:25.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6548600832 bytes 2026-02-19T20:25:25.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6613695488 bytes 2026-02-19T20:25:25.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6640995328 bytes 2026-02-19T20:25:26.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6711581696 bytes 2026-02-19T20:25:26.056 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6733550592 bytes 2026-02-19T20:25:26.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6827953152 bytes 2026-02-19T20:25:26.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6861941760 bytes 2026-02-19T20:25:26.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6892082176 bytes 2026-02-19T20:25:26.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 6909844480 bytes 2026-02-19T20:25:26.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 6965356544 bytes 2026-02-19T20:25:26.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 6995542016 bytes 2026-02-19T20:25:26.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7032485888 bytes 2026-02-19T20:25:26.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7100086272 bytes 2026-02-19T20:25:26.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7174156288 bytes 2026-02-19T20:25:26.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7186393088 bytes 2026-02-19T20:25:26.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7211545600 bytes 2026-02-19T20:25:26.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7256569856 bytes 2026-02-19T20:25:26.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7324284928 bytes 2026-02-19T20:25:26.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7388778496 bytes 2026-02-19T20:25:26.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7416083456 bytes 2026-02-19T20:25:27.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7461144576 bytes 2026-02-19T20:25:27.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7466627072 bytes 2026-02-19T20:25:27.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7529802752 bytes 2026-02-19T20:25:27.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7588959232 bytes 2026-02-19T20:25:27.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7648564224 bytes 2026-02-19T20:25:27.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7654181888 bytes 2026-02-19T20:25:27.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7689605120 bytes 2026-02-19T20:25:27.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7746536448 bytes 2026-02-19T20:25:27.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7814014976 bytes 2026-02-19T20:25:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7840555008 bytes 2026-02-19T20:25:27.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 7883323392 bytes 2026-02-19T20:25:27.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 7923722240 bytes 2026-02-19T20:25:27.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 7939829760 bytes 2026-02-19T20:25:27.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 7950184448 bytes 2026-02-19T20:25:27.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8003377152 bytes 2026-02-19T20:25:27.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8078441472 bytes 2026-02-19T20:25:27.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8100341760 bytes 2026-02-19T20:25:28.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8160897024 bytes 2026-02-19T20:25:28.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8199234560 bytes 2026-02-19T20:25:28.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8291002368 bytes 2026-02-19T20:25:28.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8348531712 bytes 2026-02-19T20:25:28.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8397829120 bytes 2026-02-19T20:25:28.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8448056320 bytes 2026-02-19T20:25:28.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8496546816 bytes 2026-02-19T20:25:28.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8566630400 bytes 2026-02-19T20:25:28.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:25:28.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:25:28.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:25:28.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:28.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:25:28.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_40 2026-02-19T20:25:28.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_40 snapshot 128M 2026-02-19T20:25:28.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_40 128M 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:28.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:28.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:28.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:25:28.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:28.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:25:28.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:25:28.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:25:28.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:28.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_40 2026-02-19T20:25:28.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:25:28.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:28.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:25:28.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:28.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_40 snapshot 2026-02-19T20:25:28.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:28.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:28.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:25:28.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:28.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_40 snapshot 2026-02-19T20:25:29.309 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.31853792481259413) > weight (0.25) 2026-02-19T20:25:29.310 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.35007468657256446) > weight (0.25) 2026-02-19T20:25:29.310 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2026-02-19T20:25:29.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2026-02-19T20:25:29.310 INFO:tasks.rbd_mirror_thrash:waiting for 5.143122821054192 secs before reviving daemons 2026-02-19T20:25:29.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:25:29.309+0000 7ff16b46d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 59433) UID: 1000 2026-02-19T20:25:29.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:25:29.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_40 2026-02-19T20:25:29.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_40': 2026-02-19T20:25:29.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:25:29.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:25:29.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:25:29.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 145d1e4b9b3 2026-02-19T20:25:29.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.145d1e4b9b3 2026-02-19T20:25:29.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:25:29.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:25:29.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:25:29.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:25:29.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:28 2026 2026-02-19T20:25:29.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:28 2026 2026-02-19T20:25:29.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:28 2026 2026-02-19T20:25:29.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:25:29.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:25:29.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 1f232c6e-6e25-4b6b-89c0-3442feb4127a 2026-02-19T20:25:29.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:25:29.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:25:29.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_40 2026-02-19T20:25:29.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:29.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:29.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:25:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:25:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:25:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:25:29.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_40 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:25:29.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 149398528 bytes 2026-02-19T20:25:29.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 192021504 bytes 2026-02-19T20:25:29.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 247970816 bytes 2026-02-19T20:25:29.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 265083904 bytes 2026-02-19T20:25:29.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 287003648 bytes 2026-02-19T20:25:30.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 309091328 bytes 2026-02-19T20:25:30.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 366710784 bytes 2026-02-19T20:25:30.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 459023360 bytes 2026-02-19T20:25:30.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 521742336 bytes 2026-02-19T20:25:30.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 569795584 bytes 2026-02-19T20:25:30.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 606786560 bytes 2026-02-19T20:25:30.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 631311360 bytes 2026-02-19T20:25:30.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 656439296 bytes 2026-02-19T20:25:30.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 670263296 bytes 2026-02-19T20:25:30.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 716756992 bytes 2026-02-19T20:25:30.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 743454720 bytes 2026-02-19T20:25:30.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 834915328 bytes 2026-02-19T20:25:30.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 896328704 bytes 2026-02-19T20:25:30.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 1008776192 bytes 2026-02-19T20:25:30.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 1065479168 bytes 2026-02-19T20:25:31.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1122727936 bytes 2026-02-19T20:25:31.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1144919040 bytes 2026-02-19T20:25:31.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1189455872 bytes 2026-02-19T20:25:31.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1201942528 bytes 2026-02-19T20:25:31.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1233069056 bytes 2026-02-19T20:25:31.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1277878272 bytes 2026-02-19T20:25:31.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1286050816 bytes 2026-02-19T20:25:31.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1341661184 bytes 2026-02-19T20:25:31.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1393562624 bytes 2026-02-19T20:25:31.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1420170240 bytes 2026-02-19T20:25:31.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1496808448 bytes 2026-02-19T20:25:31.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1568518144 bytes 2026-02-19T20:25:31.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1619496960 bytes 2026-02-19T20:25:31.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1654099968 bytes 2026-02-19T20:25:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1673938944 bytes 2026-02-19T20:25:31.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1746936832 bytes 2026-02-19T20:25:32.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1844598784 bytes 2026-02-19T20:25:32.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1901274112 bytes 2026-02-19T20:25:32.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1930296320 bytes 2026-02-19T20:25:32.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1979012096 bytes 2026-02-19T20:25:32.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 2093773824 bytes 2026-02-19T20:25:32.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 2125794304 bytes 2026-02-19T20:25:32.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2168356864 bytes 2026-02-19T20:25:32.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2250441728 bytes 2026-02-19T20:25:32.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2343737344 bytes 2026-02-19T20:25:32.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2394804224 bytes 2026-02-19T20:25:32.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2437437440 bytes 2026-02-19T20:25:32.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2470197248 bytes 2026-02-19T20:25:32.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2517730304 bytes 2026-02-19T20:25:32.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2543055872 bytes 2026-02-19T20:25:32.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2557201408 bytes 2026-02-19T20:25:32.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2588678144 bytes 2026-02-19T20:25:33.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2671753216 bytes 2026-02-19T20:25:33.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2693698560 bytes 2026-02-19T20:25:33.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2753676288 bytes 2026-02-19T20:25:33.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2789786624 bytes 2026-02-19T20:25:33.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2857190400 bytes 2026-02-19T20:25:33.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2908244992 bytes 2026-02-19T20:25:33.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2909939712 bytes 2026-02-19T20:25:33.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2957070336 bytes 2026-02-19T20:25:33.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 3008192512 bytes 2026-02-19T20:25:33.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 3095192576 bytes 2026-02-19T20:25:33.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 3161262080 bytes 2026-02-19T20:25:33.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 3209421824 bytes 2026-02-19T20:25:33.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3302784000 bytes 2026-02-19T20:25:33.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3347105792 bytes 2026-02-19T20:25:33.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3374679040 bytes 2026-02-19T20:25:34.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3416198144 bytes 2026-02-19T20:25:34.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3470302208 bytes 2026-02-19T20:25:34.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3488584704 bytes 2026-02-19T20:25:34.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3542181888 bytes 2026-02-19T20:25:34.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3585372160 bytes 2026-02-19T20:25:34.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3607549952 bytes 2026-02-19T20:25:34.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3640368128 bytes 2026-02-19T20:25:34.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3677366272 bytes 2026-02-19T20:25:34.454 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2026-02-19T20:25:34.454 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:25:34.455 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2026-02-19T20:25:34.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2026-02-19T20:25:34.455 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2026-02-19T20:25:34.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2026-02-19T20:25:34.459 INFO:tasks.rbd_mirror_thrash:waiting for 96.6355695537324 secs before thrashing 2026-02-19T20:25:34.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3747747840 bytes 2026-02-19T20:25:34.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3810951168 bytes 2026-02-19T20:25:34.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3864147968 bytes 2026-02-19T20:25:34.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3886860288 bytes 2026-02-19T20:25:34.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3924362240 bytes 2026-02-19T20:25:34.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3975732224 bytes 2026-02-19T20:25:34.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 4019305472 bytes 2026-02-19T20:25:34.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 4051514368 bytes 2026-02-19T20:25:34.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 4098751488 bytes 2026-02-19T20:25:35.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 4115569664 bytes 2026-02-19T20:25:35.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 4146330624 bytes 2026-02-19T20:25:35.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4181705728 bytes 2026-02-19T20:25:35.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4207229952 bytes 2026-02-19T20:25:35.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4218331136 bytes 2026-02-19T20:25:35.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4232013824 bytes 2026-02-19T20:25:35.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4310851584 bytes 2026-02-19T20:25:35.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4360348672 bytes 2026-02-19T20:25:35.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4390266880 bytes 2026-02-19T20:25:35.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4403378176 bytes 2026-02-19T20:25:35.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4416372736 bytes 2026-02-19T20:25:35.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4421402624 bytes 2026-02-19T20:25:35.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4491395072 bytes 2026-02-19T20:25:35.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4544483328 bytes 2026-02-19T20:25:35.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4637618176 bytes 2026-02-19T20:25:35.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4656116736 bytes 2026-02-19T20:25:35.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4688120832 bytes 2026-02-19T20:25:35.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4731276288 bytes 2026-02-19T20:25:36.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4839395328 bytes 2026-02-19T20:25:36.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4859322368 bytes 2026-02-19T20:25:36.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4891378688 bytes 2026-02-19T20:25:36.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4937374720 bytes 2026-02-19T20:25:36.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4991306752 bytes 2026-02-19T20:25:36.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 5041696768 bytes 2026-02-19T20:25:36.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 5069152256 bytes 2026-02-19T20:25:36.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5091100672 bytes 2026-02-19T20:25:36.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5136162816 bytes 2026-02-19T20:25:36.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5200008192 bytes 2026-02-19T20:25:36.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5237867520 bytes 2026-02-19T20:25:36.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5334894592 bytes 2026-02-19T20:25:36.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5352143872 bytes 2026-02-19T20:25:36.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5380718592 bytes 2026-02-19T20:25:36.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5411970048 bytes 2026-02-19T20:25:37.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5503555584 bytes 2026-02-19T20:25:37.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5552707584 bytes 2026-02-19T20:25:37.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5607318528 bytes 2026-02-19T20:25:37.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5624314880 bytes 2026-02-19T20:25:37.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5650580480 bytes 2026-02-19T20:25:37.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5722248192 bytes 2026-02-19T20:25:37.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5740581888 bytes 2026-02-19T20:25:37.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5753556992 bytes 2026-02-19T20:25:37.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5765075968 bytes 2026-02-19T20:25:37.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5819750400 bytes 2026-02-19T20:25:37.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5826263040 bytes 2026-02-19T20:25:37.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5855253504 bytes 2026-02-19T20:25:37.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5943973888 bytes 2026-02-19T20:25:37.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 6071200768 bytes 2026-02-19T20:25:38.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 6094418944 bytes 2026-02-19T20:25:38.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 6106900480 bytes 2026-02-19T20:25:38.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6149978112 bytes 2026-02-19T20:25:38.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6182632448 bytes 2026-02-19T20:25:38.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6233232384 bytes 2026-02-19T20:25:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6268710912 bytes 2026-02-19T20:25:38.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6291006464 bytes 2026-02-19T20:25:38.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6298726400 bytes 2026-02-19T20:25:38.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6304942080 bytes 2026-02-19T20:25:38.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6384876544 bytes 2026-02-19T20:25:38.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6427379712 bytes 2026-02-19T20:25:38.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6467392512 bytes 2026-02-19T20:25:38.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6534236160 bytes 2026-02-19T20:25:38.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6578407424 bytes 2026-02-19T20:25:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6594249728 bytes 2026-02-19T20:25:38.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6650691584 bytes 2026-02-19T20:25:38.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6669530112 bytes 2026-02-19T20:25:38.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6683239424 bytes 2026-02-19T20:25:39.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6704718848 bytes 2026-02-19T20:25:39.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6745026560 bytes 2026-02-19T20:25:39.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6798835712 bytes 2026-02-19T20:25:39.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6845805568 bytes 2026-02-19T20:25:39.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6916657152 bytes 2026-02-19T20:25:39.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6977131520 bytes 2026-02-19T20:25:39.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7057969152 bytes 2026-02-19T20:25:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7129084928 bytes 2026-02-19T20:25:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:25:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:25:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:25:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:25:39.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_41 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_41 snapshot 128M 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_41 128M 2026-02-19T20:25:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_41 2026-02-19T20:25:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:25:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:25:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_41 snapshot 2026-02-19T20:25:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:25:39.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:39.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_41 snapshot 2026-02-19T20:25:40.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:25:40.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_41 2026-02-19T20:25:40.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_41': 2026-02-19T20:25:40.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 146fb2e85ce2 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.146fb2e85ce2 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:39 2026 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:39 2026 2026-02-19T20:25:40.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:39 2026 2026-02-19T20:25:40.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:25:40.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:25:40.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: ec6e3c47-0016-414a-92e7-6ab7bcdd379d 2026-02-19T20:25:40.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:25:40.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:25:40.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_41 2026-02-19T20:25:40.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:40.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:40.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:25:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:25:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=11 2026-02-19T20:25:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:25:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_41 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:25:40.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 55993344 bytes 2026-02-19T20:25:40.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 146222080 bytes 2026-02-19T20:25:40.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 157604864 bytes 2026-02-19T20:25:40.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 176887808 bytes 2026-02-19T20:25:40.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 225048576 bytes 2026-02-19T20:25:41.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 294144000 bytes 2026-02-19T20:25:41.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 325965824 bytes 2026-02-19T20:25:41.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 366412800 bytes 2026-02-19T20:25:41.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 432773120 bytes 2026-02-19T20:25:41.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 444959744 bytes 2026-02-19T20:25:41.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 472361984 bytes 2026-02-19T20:25:41.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 472974336 bytes 2026-02-19T20:25:41.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 473591808 bytes 2026-02-19T20:25:41.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 540477440 bytes 2026-02-19T20:25:41.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 586302464 bytes 2026-02-19T20:25:41.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 643632128 bytes 2026-02-19T20:25:41.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 696009728 bytes 2026-02-19T20:25:41.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 773275648 bytes 2026-02-19T20:25:41.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 813919232 bytes 2026-02-19T20:25:41.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 825967616 bytes 2026-02-19T20:25:42.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 861323264 bytes 2026-02-19T20:25:42.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 929042432 bytes 2026-02-19T20:25:42.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 961230848 bytes 2026-02-19T20:25:42.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1008530432 bytes 2026-02-19T20:25:42.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1029656576 bytes 2026-02-19T20:25:42.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1063768064 bytes 2026-02-19T20:25:42.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1114397696 bytes 2026-02-19T20:25:42.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1149105152 bytes 2026-02-19T20:25:42.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1168043008 bytes 2026-02-19T20:25:42.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1202177024 bytes 2026-02-19T20:25:42.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1260642304 bytes 2026-02-19T20:25:42.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1293263872 bytes 2026-02-19T20:25:42.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1344462848 bytes 2026-02-19T20:25:42.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1441624064 bytes 2026-02-19T20:25:42.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1442691072 bytes 2026-02-19T20:25:43.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1529036800 bytes 2026-02-19T20:25:43.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1586093056 bytes 2026-02-19T20:25:43.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1594128384 bytes 2026-02-19T20:25:43.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1654495232 bytes 2026-02-19T20:25:43.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1725987840 bytes 2026-02-19T20:25:43.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1817772032 bytes 2026-02-19T20:25:43.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1867646976 bytes 2026-02-19T20:25:44.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1974985728 bytes 2026-02-19T20:25:44.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2018989056 bytes 2026-02-19T20:25:44.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2091990016 bytes 2026-02-19T20:25:44.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2146690048 bytes 2026-02-19T20:25:44.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2147326976 bytes 2026-02-19T20:25:44.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2173969408 bytes 2026-02-19T20:25:44.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2256542720 bytes 2026-02-19T20:25:44.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2275048448 bytes 2026-02-19T20:25:44.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2312800256 bytes 2026-02-19T20:25:44.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2327298048 bytes 2026-02-19T20:25:44.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2361623552 bytes 2026-02-19T20:25:44.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2372092928 bytes 2026-02-19T20:25:44.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2455225344 bytes 2026-02-19T20:25:44.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2482228224 bytes 2026-02-19T20:25:44.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2527045632 bytes 2026-02-19T20:25:44.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2555586560 bytes 2026-02-19T20:25:45.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2616631296 bytes 2026-02-19T20:25:45.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2651502592 bytes 2026-02-19T20:25:45.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2707772416 bytes 2026-02-19T20:25:45.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2712653824 bytes 2026-02-19T20:25:45.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2832519168 bytes 2026-02-19T20:25:45.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2844780544 bytes 2026-02-19T20:25:45.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2866163712 bytes 2026-02-19T20:25:45.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2866766848 bytes 2026-02-19T20:25:45.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2878289920 bytes 2026-02-19T20:25:45.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2907832320 bytes 2026-02-19T20:25:45.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2926087168 bytes 2026-02-19T20:25:45.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2968469504 bytes 2026-02-19T20:25:45.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3040278528 bytes 2026-02-19T20:25:45.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3068518400 bytes 2026-02-19T20:25:45.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3088485376 bytes 2026-02-19T20:25:45.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3098804224 bytes 2026-02-19T20:25:45.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3118580736 bytes 2026-02-19T20:25:46.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3184524288 bytes 2026-02-19T20:25:46.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3212090368 bytes 2026-02-19T20:25:46.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3257706496 bytes 2026-02-19T20:25:46.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3313276928 bytes 2026-02-19T20:25:46.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3313887232 bytes 2026-02-19T20:25:46.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3333319680 bytes 2026-02-19T20:25:46.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3343890432 bytes 2026-02-19T20:25:46.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3393508352 bytes 2026-02-19T20:25:46.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3450108928 bytes 2026-02-19T20:25:46.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3453596672 bytes 2026-02-19T20:25:46.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3490874368 bytes 2026-02-19T20:25:46.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3540640768 bytes 2026-02-19T20:25:46.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3576819712 bytes 2026-02-19T20:25:46.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3654213632 bytes 2026-02-19T20:25:46.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3664871424 bytes 2026-02-19T20:25:46.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3677618176 bytes 2026-02-19T20:25:46.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3737720832 bytes 2026-02-19T20:25:46.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3777142784 bytes 2026-02-19T20:25:47.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3814654976 bytes 2026-02-19T20:25:47.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3861250048 bytes 2026-02-19T20:25:47.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3925628928 bytes 2026-02-19T20:25:47.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4002756608 bytes 2026-02-19T20:25:47.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4007065600 bytes 2026-02-19T20:25:47.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4032579584 bytes 2026-02-19T20:25:47.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4033223680 bytes 2026-02-19T20:25:47.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4052003840 bytes 2026-02-19T20:25:47.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4075153408 bytes 2026-02-19T20:25:47.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4097209344 bytes 2026-02-19T20:25:47.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4127864832 bytes 2026-02-19T20:25:47.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4211312640 bytes 2026-02-19T20:25:47.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4261193728 bytes 2026-02-19T20:25:47.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4272672768 bytes 2026-02-19T20:25:47.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4354626560 bytes 2026-02-19T20:25:48.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4385637376 bytes 2026-02-19T20:25:48.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4424642560 bytes 2026-02-19T20:25:48.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4486476800 bytes 2026-02-19T20:25:48.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4487078912 bytes 2026-02-19T20:25:48.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4537915392 bytes 2026-02-19T20:25:48.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4548988928 bytes 2026-02-19T20:25:48.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4626264064 bytes 2026-02-19T20:25:48.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4662068224 bytes 2026-02-19T20:25:48.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4726842368 bytes 2026-02-19T20:25:48.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4760973312 bytes 2026-02-19T20:25:48.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4769047552 bytes 2026-02-19T20:25:48.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4814513152 bytes 2026-02-19T20:25:48.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4870917120 bytes 2026-02-19T20:25:48.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4915063808 bytes 2026-02-19T20:25:48.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4941467648 bytes 2026-02-19T20:25:48.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4956166144 bytes 2026-02-19T20:25:48.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 4977878016 bytes 2026-02-19T20:25:49.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5056665600 bytes 2026-02-19T20:25:49.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5128900608 bytes 2026-02-19T20:25:49.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5199193088 bytes 2026-02-19T20:25:49.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5201646592 bytes 2026-02-19T20:25:49.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5268597760 bytes 2026-02-19T20:25:49.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5327222784 bytes 2026-02-19T20:25:49.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5369257984 bytes 2026-02-19T20:25:49.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5439204352 bytes 2026-02-19T20:25:49.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5461075968 bytes 2026-02-19T20:25:49.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5489582080 bytes 2026-02-19T20:25:49.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5513832448 bytes 2026-02-19T20:25:49.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5590535168 bytes 2026-02-19T20:25:49.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5650158592 bytes 2026-02-19T20:25:50.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5728096256 bytes 2026-02-19T20:25:50.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5766301696 bytes 2026-02-19T20:25:50.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5819791360 bytes 2026-02-19T20:25:50.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5837023232 bytes 2026-02-19T20:25:50.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5902118912 bytes 2026-02-19T20:25:50.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6028171264 bytes 2026-02-19T20:25:50.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6045329408 bytes 2026-02-19T20:25:50.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6045953024 bytes 2026-02-19T20:25:50.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6106667008 bytes 2026-02-19T20:25:50.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6151748608 bytes 2026-02-19T20:25:50.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6167851008 bytes 2026-02-19T20:25:50.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6202266624 bytes 2026-02-19T20:25:50.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6260003840 bytes 2026-02-19T20:25:50.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6282933248 bytes 2026-02-19T20:25:50.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6294473728 bytes 2026-02-19T20:25:50.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6318025728 bytes 2026-02-19T20:25:50.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6363674624 bytes 2026-02-19T20:25:51.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6374126592 bytes 2026-02-19T20:25:51.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6427252736 bytes 2026-02-19T20:25:51.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6482205696 bytes 2026-02-19T20:25:51.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6504667136 bytes 2026-02-19T20:25:51.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6531578880 bytes 2026-02-19T20:25:51.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6551355392 bytes 2026-02-19T20:25:51.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6577961984 bytes 2026-02-19T20:25:51.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6599950336 bytes 2026-02-19T20:25:51.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6632287232 bytes 2026-02-19T20:25:51.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6715208704 bytes 2026-02-19T20:25:51.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6733255680 bytes 2026-02-19T20:25:51.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:25:51.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:25:51.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:25:51.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:51.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:25:51.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_42 2026-02-19T20:25:51.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_42 snapshot 128M 2026-02-19T20:25:51.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:51.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_42 128M 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:51.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:51.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:25:51.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:51.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:25:51.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:25:51.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:25:51.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:25:51.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_42 2026-02-19T20:25:51.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:25:51.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:51.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:25:51.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:25:51.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_42 snapshot 2026-02-19T20:25:51.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:51.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:51.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:25:51.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:25:51.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_42 snapshot 2026-02-19T20:25:52.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:25:52.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_42 2026-02-19T20:25:52.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_42': 2026-02-19T20:25:52.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:25:52.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 147b6d85f667 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.147b6d85f667 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:51 2026 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:51 2026 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:51 2026 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 181745d0-d881-4b58-8036-60e15d40ce25 2026-02-19T20:25:52.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:25:52.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:25:52.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_42 2026-02-19T20:25:52.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:25:52.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:25:52.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:25:52.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:25:52.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:25:52.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:25:52.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_42 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:25:52.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 52001792 bytes 2026-02-19T20:25:52.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 112969728 bytes 2026-02-19T20:25:52.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 126420992 bytes 2026-02-19T20:25:52.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 182490112 bytes 2026-02-19T20:25:53.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 233725952 bytes 2026-02-19T20:25:53.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 389253120 bytes 2026-02-19T20:25:53.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 412649472 bytes 2026-02-19T20:25:53.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 415303680 bytes 2026-02-19T20:25:53.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 465012736 bytes 2026-02-19T20:25:53.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 471469056 bytes 2026-02-19T20:25:53.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 526148608 bytes 2026-02-19T20:25:53.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 566434816 bytes 2026-02-19T20:25:53.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 617230336 bytes 2026-02-19T20:25:53.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 622162944 bytes 2026-02-19T20:25:53.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 654151680 bytes 2026-02-19T20:25:53.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 677935104 bytes 2026-02-19T20:25:53.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 707812352 bytes 2026-02-19T20:25:53.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 757323776 bytes 2026-02-19T20:25:53.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 822914048 bytes 2026-02-19T20:25:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 833215488 bytes 2026-02-19T20:25:53.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 869373952 bytes 2026-02-19T20:25:54.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 917362688 bytes 2026-02-19T20:25:54.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 959622144 bytes 2026-02-19T20:25:54.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 960244736 bytes 2026-02-19T20:25:54.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 991132672 bytes 2026-02-19T20:25:54.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1039964160 bytes 2026-02-19T20:25:54.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1095181312 bytes 2026-02-19T20:25:54.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1110257664 bytes 2026-02-19T20:25:54.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1152165888 bytes 2026-02-19T20:25:54.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1220868096 bytes 2026-02-19T20:25:54.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1272608768 bytes 2026-02-19T20:25:54.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1296354304 bytes 2026-02-19T20:25:54.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1331927040 bytes 2026-02-19T20:25:54.762 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1411211264 bytes 2026-02-19T20:25:54.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1456302080 bytes 2026-02-19T20:25:54.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1490139136 bytes 2026-02-19T20:25:54.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1564144640 bytes 2026-02-19T20:25:55.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1621214208 bytes 2026-02-19T20:25:55.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1700345856 bytes 2026-02-19T20:25:55.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1711747072 bytes 2026-02-19T20:25:55.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1777423360 bytes 2026-02-19T20:25:55.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1800455168 bytes 2026-02-19T20:25:55.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1824889856 bytes 2026-02-19T20:25:55.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1951337472 bytes 2026-02-19T20:25:55.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2006530048 bytes 2026-02-19T20:25:55.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2039506944 bytes 2026-02-19T20:25:55.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2098677760 bytes 2026-02-19T20:25:55.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2131888128 bytes 2026-02-19T20:25:55.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2177067008 bytes 2026-02-19T20:25:55.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2195196928 bytes 2026-02-19T20:25:55.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2256707584 bytes 2026-02-19T20:25:55.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2345916416 bytes 2026-02-19T20:25:56.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2379601920 bytes 2026-02-19T20:25:56.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2382410752 bytes 2026-02-19T20:25:56.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2452830208 bytes 2026-02-19T20:25:56.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2476864512 bytes 2026-02-19T20:25:56.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2527859712 bytes 2026-02-19T20:25:56.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2571853824 bytes 2026-02-19T20:25:56.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2626654208 bytes 2026-02-19T20:25:56.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2737989632 bytes 2026-02-19T20:25:56.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2749211648 bytes 2026-02-19T20:25:56.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2863706112 bytes 2026-02-19T20:25:56.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2938844160 bytes 2026-02-19T20:25:56.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2998870016 bytes 2026-02-19T20:25:56.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3022851072 bytes 2026-02-19T20:25:56.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3028876288 bytes 2026-02-19T20:25:56.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3072746496 bytes 2026-02-19T20:25:57.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3103872000 bytes 2026-02-19T20:25:57.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3153168384 bytes 2026-02-19T20:25:57.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3167875072 bytes 2026-02-19T20:25:57.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3305881600 bytes 2026-02-19T20:25:57.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3332712448 bytes 2026-02-19T20:25:57.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3361110016 bytes 2026-02-19T20:25:57.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3407938560 bytes 2026-02-19T20:25:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3435744256 bytes 2026-02-19T20:25:57.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3452858368 bytes 2026-02-19T20:25:57.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3476838400 bytes 2026-02-19T20:25:57.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3529758720 bytes 2026-02-19T20:25:57.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3568115712 bytes 2026-02-19T20:25:57.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3663305728 bytes 2026-02-19T20:25:57.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3778328576 bytes 2026-02-19T20:25:57.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3828350976 bytes 2026-02-19T20:25:58.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3919747072 bytes 2026-02-19T20:25:58.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3939867648 bytes 2026-02-19T20:25:58.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3951581184 bytes 2026-02-19T20:25:58.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 4007369728 bytes 2026-02-19T20:25:58.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4043568128 bytes 2026-02-19T20:25:58.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4093699072 bytes 2026-02-19T20:25:58.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4118410240 bytes 2026-02-19T20:25:58.538 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4233864192 bytes 2026-02-19T20:25:58.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4250187776 bytes 2026-02-19T20:25:58.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4297054208 bytes 2026-02-19T20:25:58.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4406040576 bytes 2026-02-19T20:25:58.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4442370048 bytes 2026-02-19T20:25:58.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4474964992 bytes 2026-02-19T20:25:58.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4539742208 bytes 2026-02-19T20:25:59.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4556084224 bytes 2026-02-19T20:25:59.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4556708864 bytes 2026-02-19T20:25:59.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4595602432 bytes 2026-02-19T20:25:59.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4637675520 bytes 2026-02-19T20:25:59.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4681237504 bytes 2026-02-19T20:25:59.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4702974976 bytes 2026-02-19T20:25:59.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4729331712 bytes 2026-02-19T20:25:59.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4777806848 bytes 2026-02-19T20:25:59.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4867933184 bytes 2026-02-19T20:25:59.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4904892416 bytes 2026-02-19T20:25:59.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4933072896 bytes 2026-02-19T20:25:59.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4971762688 bytes 2026-02-19T20:25:59.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 5017060352 bytes 2026-02-19T20:25:59.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5040001024 bytes 2026-02-19T20:25:59.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5061266432 bytes 2026-02-19T20:25:59.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5062755328 bytes 2026-02-19T20:25:59.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5107801088 bytes 2026-02-19T20:25:59.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5144182784 bytes 2026-02-19T20:25:59.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5156321280 bytes 2026-02-19T20:26:00.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5254770688 bytes 2026-02-19T20:26:00.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5310747648 bytes 2026-02-19T20:26:00.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5362871296 bytes 2026-02-19T20:26:00.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5390918656 bytes 2026-02-19T20:26:00.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5410692096 bytes 2026-02-19T20:26:00.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5413096448 bytes 2026-02-19T20:26:00.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5436855296 bytes 2026-02-19T20:26:00.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5500785664 bytes 2026-02-19T20:26:00.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5535055872 bytes 2026-02-19T20:26:00.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5581182976 bytes 2026-02-19T20:26:00.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5596547072 bytes 2026-02-19T20:26:00.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5637780480 bytes 2026-02-19T20:26:00.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5711699968 bytes 2026-02-19T20:26:00.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5732966400 bytes 2026-02-19T20:26:00.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5779800064 bytes 2026-02-19T20:26:01.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5902074880 bytes 2026-02-19T20:26:01.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5926523904 bytes 2026-02-19T20:26:01.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5953889280 bytes 2026-02-19T20:26:01.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5971111936 bytes 2026-02-19T20:26:01.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6020100096 bytes 2026-02-19T20:26:01.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6047253504 bytes 2026-02-19T20:26:01.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6113851392 bytes 2026-02-19T20:26:01.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6186842112 bytes 2026-02-19T20:26:01.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6215614464 bytes 2026-02-19T20:26:01.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6239205376 bytes 2026-02-19T20:26:01.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6260964352 bytes 2026-02-19T20:26:01.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6267226112 bytes 2026-02-19T20:26:01.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6294706176 bytes 2026-02-19T20:26:01.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6324512768 bytes 2026-02-19T20:26:01.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6348616704 bytes 2026-02-19T20:26:01.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6365999104 bytes 2026-02-19T20:26:01.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6436439040 bytes 2026-02-19T20:26:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6462472192 bytes 2026-02-19T20:26:02.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6518287360 bytes 2026-02-19T20:26:02.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6573381632 bytes 2026-02-19T20:26:02.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6647413760 bytes 2026-02-19T20:26:02.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6680621056 bytes 2026-02-19T20:26:02.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6694057984 bytes 2026-02-19T20:26:02.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6707981312 bytes 2026-02-19T20:26:02.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6723537920 bytes 2026-02-19T20:26:02.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6775225344 bytes 2026-02-19T20:26:02.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6810435584 bytes 2026-02-19T20:26:02.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6834387968 bytes 2026-02-19T20:26:02.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6841632768 bytes 2026-02-19T20:26:02.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6868714496 bytes 2026-02-19T20:26:02.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6893032448 bytes 2026-02-19T20:26:02.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6945137664 bytes 2026-02-19T20:26:02.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7038077952 bytes 2026-02-19T20:26:02.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7057949696 bytes 2026-02-19T20:26:03.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7111164928 bytes 2026-02-19T20:26:03.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7126494208 bytes 2026-02-19T20:26:03.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7224921088 bytes 2026-02-19T20:26:03.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7262645248 bytes 2026-02-19T20:26:03.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7342301184 bytes 2026-02-19T20:26:03.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7425569792 bytes 2026-02-19T20:26:03.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7490086912 bytes 2026-02-19T20:26:03.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7502782464 bytes 2026-02-19T20:26:03.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7549415424 bytes 2026-02-19T20:26:03.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7550028800 bytes 2026-02-19T20:26:03.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7558209536 bytes 2026-02-19T20:26:03.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7622645760 bytes 2026-02-19T20:26:03.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7669008384 bytes 2026-02-19T20:26:03.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7669635072 bytes 2026-02-19T20:26:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7707115520 bytes 2026-02-19T20:26:03.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7740738560 bytes 2026-02-19T20:26:04.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7787581440 bytes 2026-02-19T20:26:04.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7810399232 bytes 2026-02-19T20:26:04.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7810999296 bytes 2026-02-19T20:26:04.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7868324864 bytes 2026-02-19T20:26:04.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7926103040 bytes 2026-02-19T20:26:04.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7977428992 bytes 2026-02-19T20:26:04.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7978044416 bytes 2026-02-19T20:26:04.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8021032960 bytes 2026-02-19T20:26:04.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8082932736 bytes 2026-02-19T20:26:04.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8139698176 bytes 2026-02-19T20:26:04.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8166956032 bytes 2026-02-19T20:26:04.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8220281856 bytes 2026-02-19T20:26:04.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8233053184 bytes 2026-02-19T20:26:04.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8261105664 bytes 2026-02-19T20:26:04.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8280242176 bytes 2026-02-19T20:26:04.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8294177792 bytes 2026-02-19T20:26:04.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8347342848 bytes 2026-02-19T20:26:04.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8413934592 bytes 2026-02-19T20:26:05.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8492979200 bytes 2026-02-19T20:26:05.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8524077056 bytes 2026-02-19T20:26:05.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8532441088 bytes 2026-02-19T20:26:05.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8608693248 bytes 2026-02-19T20:26:05.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8698289152 bytes 2026-02-19T20:26:05.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8706790400 bytes 2026-02-19T20:26:05.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8707402752 bytes 2026-02-19T20:26:05.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8714003456 bytes 2026-02-19T20:26:05.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8759400448 bytes 2026-02-19T20:26:05.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8791445504 bytes 2026-02-19T20:26:05.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 8826833920 bytes 2026-02-19T20:26:05.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:26:05.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:26:05.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:26:05.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:05.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:26:05.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_43 2026-02-19T20:26:05.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_43 snapshot 128M 2026-02-19T20:26:05.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:05.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_43 128M 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:05.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:26:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:26:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:26:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:26:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:05.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_43 2026-02-19T20:26:05.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:26:05.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:05.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:26:05.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:05.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_43 snapshot 2026-02-19T20:26:05.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:05.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:05.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:26:05.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:05.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_43 snapshot 2026-02-19T20:26:06.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:26:06.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_43 2026-02-19T20:26:06.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_43': 2026-02-19T20:26:06.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:26:06.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:26:06.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:26:06.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 14879f1c49ba 2026-02-19T20:26:06.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.14879f1c49ba 2026-02-19T20:26:06.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:26:06.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:26:06.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:26:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:26:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:05 2026 2026-02-19T20:26:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:05 2026 2026-02-19T20:26:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:05 2026 2026-02-19T20:26:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:26:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:26:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 786b51f3-eb62-4ad3-80e9-7168fa5f0c2e 2026-02-19T20:26:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:26:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:26:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_43 2026-02-19T20:26:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:26:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:26:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=6 2026-02-19T20:26:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:26:06.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_43 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:26:06.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 56062976 bytes 2026-02-19T20:26:06.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 99329024 bytes 2026-02-19T20:26:06.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 130246656 bytes 2026-02-19T20:26:06.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 158473216 bytes 2026-02-19T20:26:06.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 183181312 bytes 2026-02-19T20:26:06.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 199491584 bytes 2026-02-19T20:26:07.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 213763072 bytes 2026-02-19T20:26:07.077 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 236692480 bytes 2026-02-19T20:26:07.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 237301760 bytes 2026-02-19T20:26:07.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 327919616 bytes 2026-02-19T20:26:07.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 353997824 bytes 2026-02-19T20:26:07.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 398754816 bytes 2026-02-19T20:26:07.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 413966336 bytes 2026-02-19T20:26:07.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 466223104 bytes 2026-02-19T20:26:07.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 511226880 bytes 2026-02-19T20:26:07.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 553902080 bytes 2026-02-19T20:26:07.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 605753344 bytes 2026-02-19T20:26:07.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 652299264 bytes 2026-02-19T20:26:07.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 668091392 bytes 2026-02-19T20:26:07.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 689001472 bytes 2026-02-19T20:26:07.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 726637568 bytes 2026-02-19T20:26:07.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 782164992 bytes 2026-02-19T20:26:07.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 819333120 bytes 2026-02-19T20:26:07.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 838326272 bytes 2026-02-19T20:26:08.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 868894720 bytes 2026-02-19T20:26:08.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 893756416 bytes 2026-02-19T20:26:08.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 964494336 bytes 2026-02-19T20:26:08.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1032948736 bytes 2026-02-19T20:26:08.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1064388608 bytes 2026-02-19T20:26:08.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1065004032 bytes 2026-02-19T20:26:08.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1100743680 bytes 2026-02-19T20:26:08.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1120328704 bytes 2026-02-19T20:26:08.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1124981760 bytes 2026-02-19T20:26:08.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1210071040 bytes 2026-02-19T20:26:08.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1245993984 bytes 2026-02-19T20:26:08.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1273924608 bytes 2026-02-19T20:26:08.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1408186368 bytes 2026-02-19T20:26:08.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1465165824 bytes 2026-02-19T20:26:08.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1475597312 bytes 2026-02-19T20:26:08.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1522480128 bytes 2026-02-19T20:26:09.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1552054272 bytes 2026-02-19T20:26:09.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1628118016 bytes 2026-02-19T20:26:09.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1684683776 bytes 2026-02-19T20:26:09.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1755150336 bytes 2026-02-19T20:26:09.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1781254144 bytes 2026-02-19T20:26:09.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1821606912 bytes 2026-02-19T20:26:09.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1896219648 bytes 2026-02-19T20:26:09.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1900364800 bytes 2026-02-19T20:26:09.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1910088704 bytes 2026-02-19T20:26:09.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1960375296 bytes 2026-02-19T20:26:09.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 1984293888 bytes 2026-02-19T20:26:09.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2039913472 bytes 2026-02-19T20:26:09.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2093729792 bytes 2026-02-19T20:26:09.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2094350336 bytes 2026-02-19T20:26:09.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2116993024 bytes 2026-02-19T20:26:09.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2174467072 bytes 2026-02-19T20:26:10.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2235297792 bytes 2026-02-19T20:26:10.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2257708032 bytes 2026-02-19T20:26:10.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2336495616 bytes 2026-02-19T20:26:10.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2399227904 bytes 2026-02-19T20:26:10.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2405790720 bytes 2026-02-19T20:26:10.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2406404096 bytes 2026-02-19T20:26:10.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2461816832 bytes 2026-02-19T20:26:10.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2490508288 bytes 2026-02-19T20:26:10.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2538564608 bytes 2026-02-19T20:26:10.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2629856256 bytes 2026-02-19T20:26:10.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2692308992 bytes 2026-02-19T20:26:10.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2716872704 bytes 2026-02-19T20:26:10.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2769904640 bytes 2026-02-19T20:26:10.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2820121600 bytes 2026-02-19T20:26:10.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2835245056 bytes 2026-02-19T20:26:11.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2859837440 bytes 2026-02-19T20:26:11.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2917933056 bytes 2026-02-19T20:26:11.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2976859136 bytes 2026-02-19T20:26:11.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3024385024 bytes 2026-02-19T20:26:11.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3056542720 bytes 2026-02-19T20:26:11.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3098140672 bytes 2026-02-19T20:26:11.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3112809472 bytes 2026-02-19T20:26:11.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3136651264 bytes 2026-02-19T20:26:11.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3160985600 bytes 2026-02-19T20:26:11.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3164356608 bytes 2026-02-19T20:26:11.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3199498240 bytes 2026-02-19T20:26:11.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3253949440 bytes 2026-02-19T20:26:11.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3291654144 bytes 2026-02-19T20:26:11.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3357879296 bytes 2026-02-19T20:26:11.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3376507904 bytes 2026-02-19T20:26:11.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3416716288 bytes 2026-02-19T20:26:11.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3444743168 bytes 2026-02-19T20:26:12.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3506516992 bytes 2026-02-19T20:26:12.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3541385216 bytes 2026-02-19T20:26:12.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3581660160 bytes 2026-02-19T20:26:12.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3627943936 bytes 2026-02-19T20:26:12.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3731418112 bytes 2026-02-19T20:26:12.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3816131584 bytes 2026-02-19T20:26:12.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3881191424 bytes 2026-02-19T20:26:12.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3900289024 bytes 2026-02-19T20:26:12.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3943425024 bytes 2026-02-19T20:26:12.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:26:12.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:26:12.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:26:12.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:12.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:26:12.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_44 2026-02-19T20:26:12.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_44 snapshot 128M 2026-02-19T20:26:12.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:12.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:12.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_44 128M 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:12.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:26:12.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:12.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:26:12.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:26:12.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:26:12.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:12.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_44 2026-02-19T20:26:12.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:26:12.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:12.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:26:12.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:12.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_44 snapshot 2026-02-19T20:26:12.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:12.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:12.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:26:12.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:12.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_44 snapshot 2026-02-19T20:26:13.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:26:13.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_44 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_44': 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 14936b72fcfd 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.14936b72fcfd 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:12 2026 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:12 2026 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:12 2026 2026-02-19T20:26:13.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:26:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:26:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 4b187764-f4c1-4c2e-832e-6fc3fbe30766 2026-02-19T20:26:13.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:26:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:26:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_44 2026-02-19T20:26:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:26:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:26:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:26:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:26:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_44 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:26:13.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 15312896 bytes 2026-02-19T20:26:13.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 88099840 bytes 2026-02-19T20:26:13.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 96753664 bytes 2026-02-19T20:26:13.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 116704256 bytes 2026-02-19T20:26:13.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 129172480 bytes 2026-02-19T20:26:13.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 183928832 bytes 2026-02-19T20:26:14.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 230858752 bytes 2026-02-19T20:26:14.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 285281280 bytes 2026-02-19T20:26:14.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 338023424 bytes 2026-02-19T20:26:14.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 401740800 bytes 2026-02-19T20:26:14.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 442384384 bytes 2026-02-19T20:26:14.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 447641600 bytes 2026-02-19T20:26:14.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 454322176 bytes 2026-02-19T20:26:14.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 500116480 bytes 2026-02-19T20:26:14.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 542558208 bytes 2026-02-19T20:26:14.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 564151296 bytes 2026-02-19T20:26:14.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 621805568 bytes 2026-02-19T20:26:14.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 670969856 bytes 2026-02-19T20:26:14.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 737019904 bytes 2026-02-19T20:26:14.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 746049536 bytes 2026-02-19T20:26:14.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 801835008 bytes 2026-02-19T20:26:14.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 854840320 bytes 2026-02-19T20:26:14.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 901684224 bytes 2026-02-19T20:26:15.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 952342528 bytes 2026-02-19T20:26:15.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 970564608 bytes 2026-02-19T20:26:15.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1001302016 bytes 2026-02-19T20:26:15.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1027244032 bytes 2026-02-19T20:26:15.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1075705856 bytes 2026-02-19T20:26:15.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1117788160 bytes 2026-02-19T20:26:15.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1132624896 bytes 2026-02-19T20:26:15.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1220293632 bytes 2026-02-19T20:26:15.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1248824320 bytes 2026-02-19T20:26:15.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1325986816 bytes 2026-02-19T20:26:15.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1368629248 bytes 2026-02-19T20:26:15.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1417739264 bytes 2026-02-19T20:26:15.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1492027392 bytes 2026-02-19T20:26:15.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1493610496 bytes 2026-02-19T20:26:15.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1523500032 bytes 2026-02-19T20:26:15.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1568615424 bytes 2026-02-19T20:26:16.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1619495936 bytes 2026-02-19T20:26:16.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1662045184 bytes 2026-02-19T20:26:16.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1688087552 bytes 2026-02-19T20:26:16.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1712584704 bytes 2026-02-19T20:26:16.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1779856384 bytes 2026-02-19T20:26:16.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1813968896 bytes 2026-02-19T20:26:16.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1842737152 bytes 2026-02-19T20:26:16.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1914142720 bytes 2026-02-19T20:26:16.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1948747776 bytes 2026-02-19T20:26:16.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1974945792 bytes 2026-02-19T20:26:16.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1984460800 bytes 2026-02-19T20:26:16.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2066113536 bytes 2026-02-19T20:26:16.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2142436352 bytes 2026-02-19T20:26:16.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2150665216 bytes 2026-02-19T20:26:16.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2204261376 bytes 2026-02-19T20:26:16.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2296822784 bytes 2026-02-19T20:26:17.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2356477952 bytes 2026-02-19T20:26:17.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2417481728 bytes 2026-02-19T20:26:17.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2454951936 bytes 2026-02-19T20:26:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2466859008 bytes 2026-02-19T20:26:17.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2470117376 bytes 2026-02-19T20:26:17.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2521833472 bytes 2026-02-19T20:26:17.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2563826688 bytes 2026-02-19T20:26:17.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2645246976 bytes 2026-02-19T20:26:17.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2777132032 bytes 2026-02-19T20:26:17.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2869806080 bytes 2026-02-19T20:26:17.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2909368320 bytes 2026-02-19T20:26:17.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2959235072 bytes 2026-02-19T20:26:17.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3023960064 bytes 2026-02-19T20:26:17.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3101763584 bytes 2026-02-19T20:26:18.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3117862912 bytes 2026-02-19T20:26:18.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3178509312 bytes 2026-02-19T20:26:18.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3179143168 bytes 2026-02-19T20:26:18.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3253467136 bytes 2026-02-19T20:26:18.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3299733504 bytes 2026-02-19T20:26:18.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3359645696 bytes 2026-02-19T20:26:18.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3422988288 bytes 2026-02-19T20:26:18.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3456865280 bytes 2026-02-19T20:26:18.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3526695936 bytes 2026-02-19T20:26:18.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3602919424 bytes 2026-02-19T20:26:18.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3677113344 bytes 2026-02-19T20:26:18.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3785240576 bytes 2026-02-19T20:26:18.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3824460800 bytes 2026-02-19T20:26:18.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3825086464 bytes 2026-02-19T20:26:18.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3866903552 bytes 2026-02-19T20:26:19.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3867536384 bytes 2026-02-19T20:26:19.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3916917760 bytes 2026-02-19T20:26:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3980532736 bytes 2026-02-19T20:26:19.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3983635456 bytes 2026-02-19T20:26:19.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4033176576 bytes 2026-02-19T20:26:19.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4056007680 bytes 2026-02-19T20:26:19.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4092138496 bytes 2026-02-19T20:26:19.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4114033664 bytes 2026-02-19T20:26:19.438 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4142485504 bytes 2026-02-19T20:26:19.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4162212864 bytes 2026-02-19T20:26:19.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4168755200 bytes 2026-02-19T20:26:19.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4187213824 bytes 2026-02-19T20:26:19.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4335381504 bytes 2026-02-19T20:26:19.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4393148416 bytes 2026-02-19T20:26:19.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4413287424 bytes 2026-02-19T20:26:19.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4453319680 bytes 2026-02-19T20:26:19.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4522121216 bytes 2026-02-19T20:26:19.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4522719232 bytes 2026-02-19T20:26:20.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4577630208 bytes 2026-02-19T20:26:20.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4602176512 bytes 2026-02-19T20:26:20.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4642078720 bytes 2026-02-19T20:26:20.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4689315840 bytes 2026-02-19T20:26:20.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4716563456 bytes 2026-02-19T20:26:20.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4783865856 bytes 2026-02-19T20:26:20.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4835340288 bytes 2026-02-19T20:26:20.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4863992832 bytes 2026-02-19T20:26:20.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4941507584 bytes 2026-02-19T20:26:20.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4985775104 bytes 2026-02-19T20:26:20.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5008567296 bytes 2026-02-19T20:26:20.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5093280768 bytes 2026-02-19T20:26:20.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5143091200 bytes 2026-02-19T20:26:20.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5193308160 bytes 2026-02-19T20:26:20.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5243081728 bytes 2026-02-19T20:26:21.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5327942656 bytes 2026-02-19T20:26:21.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5340998656 bytes 2026-02-19T20:26:21.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5391288320 bytes 2026-02-19T20:26:21.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5415091200 bytes 2026-02-19T20:26:21.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5460825088 bytes 2026-02-19T20:26:21.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5498046464 bytes 2026-02-19T20:26:21.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5521185792 bytes 2026-02-19T20:26:21.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5543326720 bytes 2026-02-19T20:26:21.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5629618176 bytes 2026-02-19T20:26:21.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5656940544 bytes 2026-02-19T20:26:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5711632384 bytes 2026-02-19T20:26:21.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5735242752 bytes 2026-02-19T20:26:21.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5778235392 bytes 2026-02-19T20:26:21.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5803107328 bytes 2026-02-19T20:26:21.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5839877120 bytes 2026-02-19T20:26:21.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5878373376 bytes 2026-02-19T20:26:21.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5937360896 bytes 2026-02-19T20:26:21.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5952485376 bytes 2026-02-19T20:26:22.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5978819584 bytes 2026-02-19T20:26:22.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6028314624 bytes 2026-02-19T20:26:22.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6150505472 bytes 2026-02-19T20:26:22.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6213142528 bytes 2026-02-19T20:26:22.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6242803712 bytes 2026-02-19T20:26:22.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6266084352 bytes 2026-02-19T20:26:22.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6349829120 bytes 2026-02-19T20:26:22.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6350444544 bytes 2026-02-19T20:26:22.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6351060992 bytes 2026-02-19T20:26:22.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6401170432 bytes 2026-02-19T20:26:22.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6488546304 bytes 2026-02-19T20:26:22.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6531342336 bytes 2026-02-19T20:26:22.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6560784384 bytes 2026-02-19T20:26:22.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6592927744 bytes 2026-02-19T20:26:22.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6614397952 bytes 2026-02-19T20:26:22.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6666270720 bytes 2026-02-19T20:26:22.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6690350080 bytes 2026-02-19T20:26:23.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6723855360 bytes 2026-02-19T20:26:23.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6741695488 bytes 2026-02-19T20:26:23.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6804702208 bytes 2026-02-19T20:26:23.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6822717440 bytes 2026-02-19T20:26:23.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6855964672 bytes 2026-02-19T20:26:23.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6876091392 bytes 2026-02-19T20:26:23.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6914234368 bytes 2026-02-19T20:26:23.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6937742336 bytes 2026-02-19T20:26:23.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6977344512 bytes 2026-02-19T20:26:23.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7033915392 bytes 2026-02-19T20:26:23.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7055712256 bytes 2026-02-19T20:26:23.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7081311232 bytes 2026-02-19T20:26:23.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7141404672 bytes 2026-02-19T20:26:23.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7198182400 bytes 2026-02-19T20:26:23.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7209504768 bytes 2026-02-19T20:26:23.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7248457728 bytes 2026-02-19T20:26:23.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7265001472 bytes 2026-02-19T20:26:23.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7326067712 bytes 2026-02-19T20:26:23.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7341670400 bytes 2026-02-19T20:26:24.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7384725504 bytes 2026-02-19T20:26:24.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7439407104 bytes 2026-02-19T20:26:24.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7501256704 bytes 2026-02-19T20:26:24.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7535041536 bytes 2026-02-19T20:26:24.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7586825216 bytes 2026-02-19T20:26:24.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7620921344 bytes 2026-02-19T20:26:24.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7628975104 bytes 2026-02-19T20:26:24.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7663238144 bytes 2026-02-19T20:26:24.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7800131584 bytes 2026-02-19T20:26:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7817035776 bytes 2026-02-19T20:26:24.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7879797760 bytes 2026-02-19T20:26:24.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7944366080 bytes 2026-02-19T20:26:24.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7958533120 bytes 2026-02-19T20:26:24.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7981826048 bytes 2026-02-19T20:26:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8003275776 bytes 2026-02-19T20:26:24.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8022680576 bytes 2026-02-19T20:26:25.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8068198400 bytes 2026-02-19T20:26:25.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8128711680 bytes 2026-02-19T20:26:25.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8203634688 bytes 2026-02-19T20:26:25.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8241582080 bytes 2026-02-19T20:26:25.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8301009920 bytes 2026-02-19T20:26:25.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8347017216 bytes 2026-02-19T20:26:25.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8347654144 bytes 2026-02-19T20:26:25.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8370852864 bytes 2026-02-19T20:26:25.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8395942912 bytes 2026-02-19T20:26:25.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8407003136 bytes 2026-02-19T20:26:25.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8454940672 bytes 2026-02-19T20:26:25.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8502764544 bytes 2026-02-19T20:26:25.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8532489216 bytes 2026-02-19T20:26:25.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8536114176 bytes 2026-02-19T20:26:25.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8599629824 bytes 2026-02-19T20:26:25.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8672433152 bytes 2026-02-19T20:26:25.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8688205824 bytes 2026-02-19T20:26:25.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8693671936 bytes 2026-02-19T20:26:26.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8747471872 bytes 2026-02-19T20:26:26.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8762706944 bytes 2026-02-19T20:26:26.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8801090560 bytes 2026-02-19T20:26:26.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 8834438144 bytes 2026-02-19T20:26:26.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 8926204928 bytes 2026-02-19T20:26:26.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 8944592896 bytes 2026-02-19T20:26:26.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 8982398976 bytes 2026-02-19T20:26:26.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 9004745728 bytes 2026-02-19T20:26:26.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 9053005824 bytes 2026-02-19T20:26:26.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 9078026240 bytes 2026-02-19T20:26:26.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 9182849024 bytes 2026-02-19T20:26:26.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:26:26.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:26:26.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:26:26.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:26.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:26:26.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_45 2026-02-19T20:26:26.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_45 snapshot 128M 2026-02-19T20:26:26.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:26.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_45 128M 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:26.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:26:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:26:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:26:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:26:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:26.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_45 2026-02-19T20:26:26.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:26:26.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:26.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:26:26.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:26.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_45 snapshot 2026-02-19T20:26:26.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:26.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:26.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:26:26.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:26.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_45 snapshot 2026-02-19T20:26:27.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:26:27.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_45 2026-02-19T20:26:27.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_45': 2026-02-19T20:26:27.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:26:27.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:26:27.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:26:27.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 149f5a5d6bf4 2026-02-19T20:26:27.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.149f5a5d6bf4 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:26 2026 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:26 2026 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:26 2026 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 11312853-10d6-49e2-a14a-c1c0aeb67353 2026-02-19T20:26:27.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:26:27.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:26:27.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_45 2026-02-19T20:26:27.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:27.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:27.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:26:27.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:26:27.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:26:27.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:26:27.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_45 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:26:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 12326912 bytes 2026-02-19T20:26:27.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 14903296 bytes 2026-02-19T20:26:27.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 73246720 bytes 2026-02-19T20:26:27.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 109993984 bytes 2026-02-19T20:26:27.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 115273728 bytes 2026-02-19T20:26:27.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 155578368 bytes 2026-02-19T20:26:28.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 235373568 bytes 2026-02-19T20:26:28.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 306163712 bytes 2026-02-19T20:26:28.196 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 376209408 bytes 2026-02-19T20:26:28.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 430019584 bytes 2026-02-19T20:26:28.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 458820608 bytes 2026-02-19T20:26:28.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 498360320 bytes 2026-02-19T20:26:28.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 535365632 bytes 2026-02-19T20:26:28.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 611237888 bytes 2026-02-19T20:26:28.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 636847104 bytes 2026-02-19T20:26:28.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 681581568 bytes 2026-02-19T20:26:28.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 773256192 bytes 2026-02-19T20:26:28.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 809109504 bytes 2026-02-19T20:26:28.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 809735168 bytes 2026-02-19T20:26:28.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 883980288 bytes 2026-02-19T20:26:28.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 888675328 bytes 2026-02-19T20:26:28.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 903662592 bytes 2026-02-19T20:26:29.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 971162624 bytes 2026-02-19T20:26:29.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1045876736 bytes 2026-02-19T20:26:29.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1115444224 bytes 2026-02-19T20:26:29.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1116061696 bytes 2026-02-19T20:26:29.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1187634176 bytes 2026-02-19T20:26:29.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1242061824 bytes 2026-02-19T20:26:29.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1265708032 bytes 2026-02-19T20:26:29.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1308545024 bytes 2026-02-19T20:26:29.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1400350720 bytes 2026-02-19T20:26:29.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1463786496 bytes 2026-02-19T20:26:29.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1487030272 bytes 2026-02-19T20:26:29.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1550823424 bytes 2026-02-19T20:26:29.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1601687552 bytes 2026-02-19T20:26:29.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1680068608 bytes 2026-02-19T20:26:30.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1738900480 bytes 2026-02-19T20:26:30.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1764992000 bytes 2026-02-19T20:26:30.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1812830208 bytes 2026-02-19T20:26:30.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1879832576 bytes 2026-02-19T20:26:30.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1936212992 bytes 2026-02-19T20:26:30.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1973145600 bytes 2026-02-19T20:26:30.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1987986432 bytes 2026-02-19T20:26:30.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2018851840 bytes 2026-02-19T20:26:30.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2040544256 bytes 2026-02-19T20:26:30.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2069891072 bytes 2026-02-19T20:26:30.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2095281152 bytes 2026-02-19T20:26:30.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2147643392 bytes 2026-02-19T20:26:30.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2219769856 bytes 2026-02-19T20:26:30.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2264601600 bytes 2026-02-19T20:26:30.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2265294848 bytes 2026-02-19T20:26:30.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2287118336 bytes 2026-02-19T20:26:30.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2335901696 bytes 2026-02-19T20:26:31.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2387559424 bytes 2026-02-19T20:26:31.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2414699520 bytes 2026-02-19T20:26:31.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2479998976 bytes 2026-02-19T20:26:31.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2530824192 bytes 2026-02-19T20:26:31.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2552179712 bytes 2026-02-19T20:26:31.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2620753920 bytes 2026-02-19T20:26:31.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2661007360 bytes 2026-02-19T20:26:31.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2701047808 bytes 2026-02-19T20:26:31.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2719583232 bytes 2026-02-19T20:26:31.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2784412672 bytes 2026-02-19T20:26:31.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2803035136 bytes 2026-02-19T20:26:31.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2807833600 bytes 2026-02-19T20:26:31.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2808431616 bytes 2026-02-19T20:26:31.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2832740352 bytes 2026-02-19T20:26:31.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2864893952 bytes 2026-02-19T20:26:31.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2925390848 bytes 2026-02-19T20:26:31.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2926009344 bytes 2026-02-19T20:26:31.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2926607360 bytes 2026-02-19T20:26:31.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2941184000 bytes 2026-02-19T20:26:31.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2962331648 bytes 2026-02-19T20:26:32.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3018057728 bytes 2026-02-19T20:26:32.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3172843520 bytes 2026-02-19T20:26:32.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3197758464 bytes 2026-02-19T20:26:32.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3236392960 bytes 2026-02-19T20:26:32.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3293761536 bytes 2026-02-19T20:26:32.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3296911360 bytes 2026-02-19T20:26:32.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3354971136 bytes 2026-02-19T20:26:32.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3377008640 bytes 2026-02-19T20:26:32.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3466628096 bytes 2026-02-19T20:26:32.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3500249088 bytes 2026-02-19T20:26:32.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3547118592 bytes 2026-02-19T20:26:32.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3650305024 bytes 2026-02-19T20:26:32.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3706760192 bytes 2026-02-19T20:26:32.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3722032128 bytes 2026-02-19T20:26:32.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3755353088 bytes 2026-02-19T20:26:33.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3789478912 bytes 2026-02-19T20:26:33.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3791885312 bytes 2026-02-19T20:26:33.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3825339392 bytes 2026-02-19T20:26:33.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3857419264 bytes 2026-02-19T20:26:33.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3936105472 bytes 2026-02-19T20:26:33.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3967237120 bytes 2026-02-19T20:26:33.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4033486848 bytes 2026-02-19T20:26:33.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4061619200 bytes 2026-02-19T20:26:33.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4062222336 bytes 2026-02-19T20:26:33.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4118202368 bytes 2026-02-19T20:26:33.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4139208704 bytes 2026-02-19T20:26:33.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4139810816 bytes 2026-02-19T20:26:33.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4167194624 bytes 2026-02-19T20:26:33.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4234245120 bytes 2026-02-19T20:26:33.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4265185280 bytes 2026-02-19T20:26:33.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4311950336 bytes 2026-02-19T20:26:33.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4312535040 bytes 2026-02-19T20:26:33.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4336598016 bytes 2026-02-19T20:26:33.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4345741312 bytes 2026-02-19T20:26:33.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4378923008 bytes 2026-02-19T20:26:34.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4396998656 bytes 2026-02-19T20:26:34.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4482956288 bytes 2026-02-19T20:26:34.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4488148992 bytes 2026-02-19T20:26:34.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4521325568 bytes 2026-02-19T20:26:34.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4558393344 bytes 2026-02-19T20:26:34.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4592840704 bytes 2026-02-19T20:26:34.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4614373376 bytes 2026-02-19T20:26:34.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4633776128 bytes 2026-02-19T20:26:34.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4677662720 bytes 2026-02-19T20:26:34.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4715359232 bytes 2026-02-19T20:26:34.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4737830912 bytes 2026-02-19T20:26:34.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4760386560 bytes 2026-02-19T20:26:34.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4782100480 bytes 2026-02-19T20:26:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4816274432 bytes 2026-02-19T20:26:34.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4840584192 bytes 2026-02-19T20:26:34.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 4870951936 bytes 2026-02-19T20:26:34.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 4878422016 bytes 2026-02-19T20:26:34.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 4965192704 bytes 2026-02-19T20:26:35.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5029264384 bytes 2026-02-19T20:26:35.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5073285120 bytes 2026-02-19T20:26:35.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5108854784 bytes 2026-02-19T20:26:35.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5137750016 bytes 2026-02-19T20:26:35.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5193186304 bytes 2026-02-19T20:26:35.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5229156352 bytes 2026-02-19T20:26:35.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5302095872 bytes 2026-02-19T20:26:35.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5314353152 bytes 2026-02-19T20:26:35.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5334289408 bytes 2026-02-19T20:26:35.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5355732992 bytes 2026-02-19T20:26:35.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5375728640 bytes 2026-02-19T20:26:35.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5387013120 bytes 2026-02-19T20:26:35.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5398446080 bytes 2026-02-19T20:26:35.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5498527744 bytes 2026-02-19T20:26:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5556794368 bytes 2026-02-19T20:26:35.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5573901312 bytes 2026-02-19T20:26:35.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5615185920 bytes 2026-02-19T20:26:36.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5646757888 bytes 2026-02-19T20:26:36.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5713014784 bytes 2026-02-19T20:26:36.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5784423424 bytes 2026-02-19T20:26:36.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5800148992 bytes 2026-02-19T20:26:36.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5831662592 bytes 2026-02-19T20:26:36.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 5926366208 bytes 2026-02-19T20:26:36.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 5991477248 bytes 2026-02-19T20:26:36.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6014110720 bytes 2026-02-19T20:26:36.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6086256640 bytes 2026-02-19T20:26:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6126622720 bytes 2026-02-19T20:26:36.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6203973632 bytes 2026-02-19T20:26:36.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6227526656 bytes 2026-02-19T20:26:36.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6297377792 bytes 2026-02-19T20:26:36.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6319863808 bytes 2026-02-19T20:26:36.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6369370112 bytes 2026-02-19T20:26:36.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6393620480 bytes 2026-02-19T20:26:37.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6431785984 bytes 2026-02-19T20:26:37.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6469584896 bytes 2026-02-19T20:26:37.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6497596416 bytes 2026-02-19T20:26:37.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6524393472 bytes 2026-02-19T20:26:37.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6550474752 bytes 2026-02-19T20:26:37.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6607903744 bytes 2026-02-19T20:26:37.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6662823936 bytes 2026-02-19T20:26:37.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6759597056 bytes 2026-02-19T20:26:37.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6851454976 bytes 2026-02-19T20:26:37.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6889918464 bytes 2026-02-19T20:26:37.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6996396032 bytes 2026-02-19T20:26:37.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7016598528 bytes 2026-02-19T20:26:37.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7074919424 bytes 2026-02-19T20:26:37.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7106984960 bytes 2026-02-19T20:26:37.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7143054336 bytes 2026-02-19T20:26:38.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7171838976 bytes 2026-02-19T20:26:38.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7184696320 bytes 2026-02-19T20:26:38.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7240164352 bytes 2026-02-19T20:26:38.196 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7308611584 bytes 2026-02-19T20:26:38.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7338681344 bytes 2026-02-19T20:26:38.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7405089792 bytes 2026-02-19T20:26:38.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7453452288 bytes 2026-02-19T20:26:38.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7454060544 bytes 2026-02-19T20:26:38.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7497326592 bytes 2026-02-19T20:26:38.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7612248064 bytes 2026-02-19T20:26:38.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7693058048 bytes 2026-02-19T20:26:38.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7725421568 bytes 2026-02-19T20:26:38.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7796818944 bytes 2026-02-19T20:26:38.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7817827328 bytes 2026-02-19T20:26:38.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7869852672 bytes 2026-02-19T20:26:38.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7874337792 bytes 2026-02-19T20:26:39.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7903879168 bytes 2026-02-19T20:26:39.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7993341952 bytes 2026-02-19T20:26:39.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8127352832 bytes 2026-02-19T20:26:39.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8163646464 bytes 2026-02-19T20:26:39.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8167831552 bytes 2026-02-19T20:26:39.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8179255296 bytes 2026-02-19T20:26:39.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8246649856 bytes 2026-02-19T20:26:39.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8278290432 bytes 2026-02-19T20:26:39.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8333933568 bytes 2026-02-19T20:26:39.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8360268800 bytes 2026-02-19T20:26:39.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8393548800 bytes 2026-02-19T20:26:39.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8457410560 bytes 2026-02-19T20:26:39.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8519050240 bytes 2026-02-19T20:26:39.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8559321088 bytes 2026-02-19T20:26:39.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8603467776 bytes 2026-02-19T20:26:40.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8639003648 bytes 2026-02-19T20:26:40.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8678621184 bytes 2026-02-19T20:26:40.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8683779072 bytes 2026-02-19T20:26:40.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 8757130240 bytes 2026-02-19T20:26:40.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 8796297216 bytes 2026-02-19T20:26:40.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 8831239168 bytes 2026-02-19T20:26:40.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 8865593344 bytes 2026-02-19T20:26:40.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 8879393792 bytes 2026-02-19T20:26:40.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 8912158720 bytes 2026-02-19T20:26:40.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 8985358336 bytes 2026-02-19T20:26:40.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 9044242432 bytes 2026-02-19T20:26:40.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:26:40.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:26:40.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:26:40.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_46 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_46 snapshot 128M 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:40.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_46 128M 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:26:40.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:40.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_46 2026-02-19T20:26:40.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:26:40.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:40.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:26:40.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:40.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_46 snapshot 2026-02-19T20:26:40.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:40.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:40.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:26:40.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:40.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_46 snapshot 2026-02-19T20:26:41.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:26:41.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_46 2026-02-19T20:26:41.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_46': 2026-02-19T20:26:41.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 14ab58760e28 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.14ab58760e28 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:40 2026 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:40 2026 2026-02-19T20:26:41.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:40 2026 2026-02-19T20:26:41.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:26:41.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:26:41.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: fa7f549b-1b88-4438-8fba-afa9db93d973 2026-02-19T20:26:41.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:26:41.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:26:41.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_46 2026-02-19T20:26:41.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:41.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:41.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:26:41.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:26:41.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=11 2026-02-19T20:26:41.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:26:41.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_46 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:26:41.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 62867456 bytes 2026-02-19T20:26:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 131425280 bytes 2026-02-19T20:26:41.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 165217280 bytes 2026-02-19T20:26:41.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 202235904 bytes 2026-02-19T20:26:41.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 251934720 bytes 2026-02-19T20:26:42.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 306662400 bytes 2026-02-19T20:26:42.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 373830656 bytes 2026-02-19T20:26:42.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 389964800 bytes 2026-02-19T20:26:42.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 464904192 bytes 2026-02-19T20:26:42.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 498181120 bytes 2026-02-19T20:26:42.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 591593472 bytes 2026-02-19T20:26:42.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 651683840 bytes 2026-02-19T20:26:42.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 686517248 bytes 2026-02-19T20:26:42.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 783462400 bytes 2026-02-19T20:26:42.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 815595520 bytes 2026-02-19T20:26:42.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 884270080 bytes 2026-02-19T20:26:42.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 906624000 bytes 2026-02-19T20:26:42.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 924075008 bytes 2026-02-19T20:26:42.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 931011584 bytes 2026-02-19T20:26:42.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 961486848 bytes 2026-02-19T20:26:43.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1016058880 bytes 2026-02-19T20:26:43.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1040949248 bytes 2026-02-19T20:26:43.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1051762688 bytes 2026-02-19T20:26:43.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1130792960 bytes 2026-02-19T20:26:43.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1148367872 bytes 2026-02-19T20:26:43.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1233133568 bytes 2026-02-19T20:26:43.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1276018688 bytes 2026-02-19T20:26:43.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1296014336 bytes 2026-02-19T20:26:43.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1311843328 bytes 2026-02-19T20:26:43.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1368894464 bytes 2026-02-19T20:26:43.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1430337536 bytes 2026-02-19T20:26:43.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1430931456 bytes 2026-02-19T20:26:43.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38059525307028197) > weight (0.25) 2026-02-19T20:26:43.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9378295606759284) > weight (0.25) 2026-02-19T20:26:43.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6601210661024701) > weight (0.25) 2026-02-19T20:26:43.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3332793250649403) > weight (0.25) 2026-02-19T20:26:43.721 INFO:tasks.rbd_mirror_thrash:waiting for 89.70887901474174 secs before thrashing 2026-02-19T20:26:43.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1494436864 bytes 2026-02-19T20:26:43.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1525819392 bytes 2026-02-19T20:26:43.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1645186048 bytes 2026-02-19T20:26:43.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1665878016 bytes 2026-02-19T20:26:44.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1778116608 bytes 2026-02-19T20:26:44.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1863358464 bytes 2026-02-19T20:26:44.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1894583296 bytes 2026-02-19T20:26:44.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1953549312 bytes 2026-02-19T20:26:44.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 2005684224 bytes 2026-02-19T20:26:44.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 2101024768 bytes 2026-02-19T20:26:44.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2154512384 bytes 2026-02-19T20:26:44.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2175942656 bytes 2026-02-19T20:26:44.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2224680960 bytes 2026-02-19T20:26:44.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2265723904 bytes 2026-02-19T20:26:44.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2338151424 bytes 2026-02-19T20:26:44.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2351073280 bytes 2026-02-19T20:26:44.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2360798208 bytes 2026-02-19T20:26:44.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2426634240 bytes 2026-02-19T20:26:45.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2486385664 bytes 2026-02-19T20:26:45.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2573409280 bytes 2026-02-19T20:26:45.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2628626432 bytes 2026-02-19T20:26:45.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2686021632 bytes 2026-02-19T20:26:45.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2777028608 bytes 2026-02-19T20:26:45.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2783522816 bytes 2026-02-19T20:26:45.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2822499328 bytes 2026-02-19T20:26:45.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2883619840 bytes 2026-02-19T20:26:45.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2901574656 bytes 2026-02-19T20:26:45.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2994258944 bytes 2026-02-19T20:26:45.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 3029257216 bytes 2026-02-19T20:26:45.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 3105713152 bytes 2026-02-19T20:26:45.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 3136601088 bytes 2026-02-19T20:26:45.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 3205128192 bytes 2026-02-19T20:26:45.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3228229632 bytes 2026-02-19T20:26:46.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3323888640 bytes 2026-02-19T20:26:46.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3367376896 bytes 2026-02-19T20:26:46.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3431831552 bytes 2026-02-19T20:26:46.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3460177920 bytes 2026-02-19T20:26:46.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3491918848 bytes 2026-02-19T20:26:46.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3492546560 bytes 2026-02-19T20:26:46.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3572050944 bytes 2026-02-19T20:26:46.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3635082240 bytes 2026-02-19T20:26:46.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3696236544 bytes 2026-02-19T20:26:46.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3712679936 bytes 2026-02-19T20:26:46.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3741415424 bytes 2026-02-19T20:26:46.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3742015488 bytes 2026-02-19T20:26:46.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3767158784 bytes 2026-02-19T20:26:46.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3840323584 bytes 2026-02-19T20:26:46.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3882517504 bytes 2026-02-19T20:26:46.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3914996736 bytes 2026-02-19T20:26:47.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 4011413504 bytes 2026-02-19T20:26:47.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 4023783424 bytes 2026-02-19T20:26:47.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 4071900160 bytes 2026-02-19T20:26:47.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 4132894720 bytes 2026-02-19T20:26:47.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 4143970304 bytes 2026-02-19T20:26:47.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 4179570688 bytes 2026-02-19T20:26:47.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4225628160 bytes 2026-02-19T20:26:47.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4290214912 bytes 2026-02-19T20:26:47.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4342718464 bytes 2026-02-19T20:26:47.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4367789056 bytes 2026-02-19T20:26:47.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4449964032 bytes 2026-02-19T20:26:47.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4505034752 bytes 2026-02-19T20:26:47.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4524170240 bytes 2026-02-19T20:26:47.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4573474816 bytes 2026-02-19T20:26:47.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4609125376 bytes 2026-02-19T20:26:48.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4666071040 bytes 2026-02-19T20:26:48.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4731787264 bytes 2026-02-19T20:26:48.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4741345280 bytes 2026-02-19T20:26:48.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4754001920 bytes 2026-02-19T20:26:48.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4778705920 bytes 2026-02-19T20:26:48.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4794770432 bytes 2026-02-19T20:26:48.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4820951040 bytes 2026-02-19T20:26:48.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4850097152 bytes 2026-02-19T20:26:48.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4888640512 bytes 2026-02-19T20:26:48.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4919663616 bytes 2026-02-19T20:26:48.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4962238464 bytes 2026-02-19T20:26:48.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4986589184 bytes 2026-02-19T20:26:48.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 5036640256 bytes 2026-02-19T20:26:48.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5132661760 bytes 2026-02-19T20:26:48.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5172346880 bytes 2026-02-19T20:26:48.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5203232768 bytes 2026-02-19T20:26:48.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5243692032 bytes 2026-02-19T20:26:49.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5296533504 bytes 2026-02-19T20:26:49.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5374599168 bytes 2026-02-19T20:26:49.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5407740928 bytes 2026-02-19T20:26:49.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5410217984 bytes 2026-02-19T20:26:49.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5467504640 bytes 2026-02-19T20:26:49.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5509425152 bytes 2026-02-19T20:26:49.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5549937664 bytes 2026-02-19T20:26:49.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5566418944 bytes 2026-02-19T20:26:49.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5567030272 bytes 2026-02-19T20:26:49.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5619779584 bytes 2026-02-19T20:26:49.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5687117824 bytes 2026-02-19T20:26:49.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5768767488 bytes 2026-02-19T20:26:49.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5826481152 bytes 2026-02-19T20:26:49.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5930061824 bytes 2026-02-19T20:26:49.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5993057280 bytes 2026-02-19T20:26:50.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 6077413376 bytes 2026-02-19T20:26:50.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 6078044160 bytes 2026-02-19T20:26:50.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 6160151552 bytes 2026-02-19T20:26:50.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 6208691200 bytes 2026-02-19T20:26:50.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 6237500416 bytes 2026-02-19T20:26:50.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6281974784 bytes 2026-02-19T20:26:50.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6356797440 bytes 2026-02-19T20:26:50.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6442098688 bytes 2026-02-19T20:26:50.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6508249088 bytes 2026-02-19T20:26:50.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6532600832 bytes 2026-02-19T20:26:50.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6611570688 bytes 2026-02-19T20:26:50.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6658229248 bytes 2026-02-19T20:26:50.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6757534720 bytes 2026-02-19T20:26:50.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6810735616 bytes 2026-02-19T20:26:50.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6815320064 bytes 2026-02-19T20:26:51.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6865146880 bytes 2026-02-19T20:26:51.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6881191936 bytes 2026-02-19T20:26:51.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6917314560 bytes 2026-02-19T20:26:51.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6959665152 bytes 2026-02-19T20:26:51.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 7019355136 bytes 2026-02-19T20:26:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 7055485952 bytes 2026-02-19T20:26:51.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 7094336512 bytes 2026-02-19T20:26:51.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 7169880064 bytes 2026-02-19T20:26:51.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 7215145984 bytes 2026-02-19T20:26:51.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 7279104000 bytes 2026-02-19T20:26:51.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 7307827200 bytes 2026-02-19T20:26:51.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 7380872192 bytes 2026-02-19T20:26:51.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7445448704 bytes 2026-02-19T20:26:51.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7509843968 bytes 2026-02-19T20:26:51.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7530449920 bytes 2026-02-19T20:26:51.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7553530880 bytes 2026-02-19T20:26:52.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7585932288 bytes 2026-02-19T20:26:52.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7627478016 bytes 2026-02-19T20:26:52.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7686035456 bytes 2026-02-19T20:26:52.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7732429824 bytes 2026-02-19T20:26:52.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7825096704 bytes 2026-02-19T20:26:52.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7861101568 bytes 2026-02-19T20:26:52.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7923641344 bytes 2026-02-19T20:26:52.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7924264960 bytes 2026-02-19T20:26:52.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7989074944 bytes 2026-02-19T20:26:52.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7999995904 bytes 2026-02-19T20:26:52.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:26:52.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:26:52.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:26:52.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:52.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:26:52.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_47 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_47 snapshot 128M 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:52.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_47 128M 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:26:52.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:26:52.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_47 2026-02-19T20:26:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:26:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:26:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:26:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_47 snapshot 2026-02-19T20:26:52.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:52.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:52.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:26:52.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:26:52.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_47 snapshot 2026-02-19T20:26:53.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:26:53.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_47 2026-02-19T20:26:53.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_47': 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 14b7ef0b6b54 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.14b7ef0b6b54 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:26:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:52 2026 2026-02-19T20:26:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:52 2026 2026-02-19T20:26:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:52 2026 2026-02-19T20:26:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:26:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:26:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 9bf92945-bae6-44d1-9712-24406b283d4b 2026-02-19T20:26:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:26:53.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:26:53.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_47 2026-02-19T20:26:53.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:26:53.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:26:53.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:26:53.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:26:53.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=9 2026-02-19T20:26:53.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:26:53.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_47 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:26:53.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 68405248 bytes 2026-02-19T20:26:53.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 98485248 bytes 2026-02-19T20:26:53.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 162802688 bytes 2026-02-19T20:26:53.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 249216000 bytes 2026-02-19T20:26:53.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 249842688 bytes 2026-02-19T20:26:54.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 275624960 bytes 2026-02-19T20:26:54.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 306948096 bytes 2026-02-19T20:26:54.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 346313728 bytes 2026-02-19T20:26:54.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 391710720 bytes 2026-02-19T20:26:54.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 414116864 bytes 2026-02-19T20:26:54.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 472129536 bytes 2026-02-19T20:26:54.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 504216576 bytes 2026-02-19T20:26:54.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 557843456 bytes 2026-02-19T20:26:54.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 612934656 bytes 2026-02-19T20:26:54.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 623431680 bytes 2026-02-19T20:26:54.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 677751808 bytes 2026-02-19T20:26:54.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 705240064 bytes 2026-02-19T20:26:54.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 709064704 bytes 2026-02-19T20:26:54.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 751875072 bytes 2026-02-19T20:26:54.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 764441600 bytes 2026-02-19T20:26:54.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 850095104 bytes 2026-02-19T20:26:54.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 891220992 bytes 2026-02-19T20:26:55.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 936656896 bytes 2026-02-19T20:26:55.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 988301312 bytes 2026-02-19T20:26:55.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1028968448 bytes 2026-02-19T20:26:55.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1030280192 bytes 2026-02-19T20:26:55.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1078379520 bytes 2026-02-19T20:26:55.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1084312576 bytes 2026-02-19T20:26:55.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1160382464 bytes 2026-02-19T20:26:55.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1222618112 bytes 2026-02-19T20:26:55.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1239584768 bytes 2026-02-19T20:26:55.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1318626304 bytes 2026-02-19T20:26:55.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1348660224 bytes 2026-02-19T20:26:55.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1402213376 bytes 2026-02-19T20:26:55.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1442569216 bytes 2026-02-19T20:26:55.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1562281984 bytes 2026-02-19T20:26:56.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1613109248 bytes 2026-02-19T20:26:56.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1733663744 bytes 2026-02-19T20:26:56.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1761038336 bytes 2026-02-19T20:26:56.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1788278784 bytes 2026-02-19T20:26:56.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1806702592 bytes 2026-02-19T20:26:56.439 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1892503552 bytes 2026-02-19T20:26:56.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1930383360 bytes 2026-02-19T20:26:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2023218176 bytes 2026-02-19T20:26:56.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2046970880 bytes 2026-02-19T20:26:56.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2078306304 bytes 2026-02-19T20:26:56.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2127143936 bytes 2026-02-19T20:26:56.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2202104832 bytes 2026-02-19T20:26:56.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2290735104 bytes 2026-02-19T20:26:57.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2371965952 bytes 2026-02-19T20:26:57.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2473464832 bytes 2026-02-19T20:26:57.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2498567168 bytes 2026-02-19T20:26:57.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2533826560 bytes 2026-02-19T20:26:57.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2566881280 bytes 2026-02-19T20:26:57.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2657384448 bytes 2026-02-19T20:26:57.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2706876416 bytes 2026-02-19T20:26:57.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2732919808 bytes 2026-02-19T20:26:57.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2780447744 bytes 2026-02-19T20:26:57.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2830586880 bytes 2026-02-19T20:26:57.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2860709888 bytes 2026-02-19T20:26:57.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2920009728 bytes 2026-02-19T20:26:57.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2983914496 bytes 2026-02-19T20:26:57.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2984520704 bytes 2026-02-19T20:26:58.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 3002925056 bytes 2026-02-19T20:26:58.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3023029248 bytes 2026-02-19T20:26:58.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3042308096 bytes 2026-02-19T20:26:58.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3118093312 bytes 2026-02-19T20:26:58.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3125791744 bytes 2026-02-19T20:26:58.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3155645440 bytes 2026-02-19T20:26:58.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3233093632 bytes 2026-02-19T20:26:58.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3292865536 bytes 2026-02-19T20:26:58.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3321408512 bytes 2026-02-19T20:26:58.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3359470592 bytes 2026-02-19T20:26:58.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3446416384 bytes 2026-02-19T20:26:58.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3547871232 bytes 2026-02-19T20:26:58.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3578144768 bytes 2026-02-19T20:26:58.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3578741760 bytes 2026-02-19T20:26:58.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3598566400 bytes 2026-02-19T20:26:59.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3645117440 bytes 2026-02-19T20:26:59.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3677037568 bytes 2026-02-19T20:26:59.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3715461120 bytes 2026-02-19T20:26:59.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3730331648 bytes 2026-02-19T20:26:59.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3774368768 bytes 2026-02-19T20:26:59.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3805519872 bytes 2026-02-19T20:26:59.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3844842496 bytes 2026-02-19T20:26:59.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3851824128 bytes 2026-02-19T20:26:59.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3875671040 bytes 2026-02-19T20:26:59.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3939939328 bytes 2026-02-19T20:26:59.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3959216128 bytes 2026-02-19T20:26:59.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4028537856 bytes 2026-02-19T20:26:59.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4080929792 bytes 2026-02-19T20:26:59.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4130220032 bytes 2026-02-19T20:26:59.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4173743104 bytes 2026-02-19T20:26:59.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4220485632 bytes 2026-02-19T20:27:00.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4320080896 bytes 2026-02-19T20:27:00.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4372042752 bytes 2026-02-19T20:27:00.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4383022080 bytes 2026-02-19T20:27:00.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4405873664 bytes 2026-02-19T20:27:00.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4476508160 bytes 2026-02-19T20:27:00.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4590706688 bytes 2026-02-19T20:27:00.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4647144448 bytes 2026-02-19T20:27:00.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4654407680 bytes 2026-02-19T20:27:00.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4701276160 bytes 2026-02-19T20:27:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4741763072 bytes 2026-02-19T20:27:00.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4755338240 bytes 2026-02-19T20:27:00.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4784988160 bytes 2026-02-19T20:27:00.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4801008640 bytes 2026-02-19T20:27:00.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4856348672 bytes 2026-02-19T20:27:00.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4860959744 bytes 2026-02-19T20:27:00.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4883023872 bytes 2026-02-19T20:27:01.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4972593152 bytes 2026-02-19T20:27:01.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5069047808 bytes 2026-02-19T20:27:01.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5085385728 bytes 2026-02-19T20:27:01.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5111750656 bytes 2026-02-19T20:27:01.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5129464832 bytes 2026-02-19T20:27:01.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5152945152 bytes 2026-02-19T20:27:01.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5177403392 bytes 2026-02-19T20:27:01.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5211847680 bytes 2026-02-19T20:27:01.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5280740352 bytes 2026-02-19T20:27:01.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5363026944 bytes 2026-02-19T20:27:01.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5393902592 bytes 2026-02-19T20:27:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5470541824 bytes 2026-02-19T20:27:01.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5500265472 bytes 2026-02-19T20:27:01.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5508215808 bytes 2026-02-19T20:27:02.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5595087872 bytes 2026-02-19T20:27:02.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5639011328 bytes 2026-02-19T20:27:02.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5662836736 bytes 2026-02-19T20:27:02.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5710971904 bytes 2026-02-19T20:27:02.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5762703360 bytes 2026-02-19T20:27:02.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5869526016 bytes 2026-02-19T20:27:02.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5936143360 bytes 2026-02-19T20:27:02.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 6001912832 bytes 2026-02-19T20:27:02.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_48 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_48 snapshot 128M 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:02.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:27:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:27:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:27:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_48 128M 2026-02-19T20:27:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:02.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:27:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:27:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:27:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:27:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:02.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_48 2026-02-19T20:27:02.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:27:02.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:02.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:27:02.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:02.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_48 snapshot 2026-02-19T20:27:02.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:02.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:02.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:27:02.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:27:02.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_48 snapshot 2026-02-19T20:27:03.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:27:03.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_48 2026-02-19T20:27:03.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_48': 2026-02-19T20:27:03.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:27:03.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:27:03.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:27:03.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 14c33899d1e3 2026-02-19T20:27:03.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.14c33899d1e3 2026-02-19T20:27:03.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:27:03.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:27:03.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:27:03.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:27:03.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:27:02 2026 2026-02-19T20:27:03.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:27:02 2026 2026-02-19T20:27:03.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:27:02 2026 2026-02-19T20:27:03.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:27:03.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:27:03.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 893ecffe-1a73-4473-a8e4-b92236e68f8f 2026-02-19T20:27:03.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:27:03.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:27:03.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_48 2026-02-19T20:27:03.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:03.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:03.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:27:03.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:27:03.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:27:03.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:27:03.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_48 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:27:03.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 28404736 bytes 2026-02-19T20:27:03.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 119767040 bytes 2026-02-19T20:27:03.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 150517760 bytes 2026-02-19T20:27:03.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 220108800 bytes 2026-02-19T20:27:03.979 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 220804096 bytes 2026-02-19T20:27:04.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 230832128 bytes 2026-02-19T20:27:04.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 269706240 bytes 2026-02-19T20:27:04.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 293641216 bytes 2026-02-19T20:27:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 350470144 bytes 2026-02-19T20:27:04.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 365671424 bytes 2026-02-19T20:27:04.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 410720256 bytes 2026-02-19T20:27:04.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 411366400 bytes 2026-02-19T20:27:04.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 424197120 bytes 2026-02-19T20:27:04.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 460907520 bytes 2026-02-19T20:27:04.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 467730432 bytes 2026-02-19T20:27:04.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 495726592 bytes 2026-02-19T20:27:04.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 550590464 bytes 2026-02-19T20:27:04.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 649676800 bytes 2026-02-19T20:27:04.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 650272768 bytes 2026-02-19T20:27:04.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 666408960 bytes 2026-02-19T20:27:04.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 693872640 bytes 2026-02-19T20:27:04.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 730748928 bytes 2026-02-19T20:27:04.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 865064960 bytes 2026-02-19T20:27:05.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 908700672 bytes 2026-02-19T20:27:05.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 948432896 bytes 2026-02-19T20:27:05.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1005800448 bytes 2026-02-19T20:27:05.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1052786688 bytes 2026-02-19T20:27:05.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1074584576 bytes 2026-02-19T20:27:05.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1112187904 bytes 2026-02-19T20:27:05.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1131309056 bytes 2026-02-19T20:27:05.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1154782208 bytes 2026-02-19T20:27:05.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1157735424 bytes 2026-02-19T20:27:05.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1255829504 bytes 2026-02-19T20:27:05.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1322328064 bytes 2026-02-19T20:27:05.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1322931200 bytes 2026-02-19T20:27:05.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1368670208 bytes 2026-02-19T20:27:05.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1369297920 bytes 2026-02-19T20:27:05.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1386851328 bytes 2026-02-19T20:27:05.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1434873856 bytes 2026-02-19T20:27:05.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1509888000 bytes 2026-02-19T20:27:05.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1538436096 bytes 2026-02-19T20:27:06.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1649529856 bytes 2026-02-19T20:27:06.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1680329728 bytes 2026-02-19T20:27:06.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1748146176 bytes 2026-02-19T20:27:06.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1817721856 bytes 2026-02-19T20:27:06.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1818339328 bytes 2026-02-19T20:27:06.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1834051584 bytes 2026-02-19T20:27:06.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1884465152 bytes 2026-02-19T20:27:06.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1920002048 bytes 2026-02-19T20:27:06.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 1950964736 bytes 2026-02-19T20:27:06.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2027163648 bytes 2026-02-19T20:27:06.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2070045696 bytes 2026-02-19T20:27:06.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2077003776 bytes 2026-02-19T20:27:06.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2099615744 bytes 2026-02-19T20:27:06.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2100232192 bytes 2026-02-19T20:27:06.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2125976576 bytes 2026-02-19T20:27:06.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2166706176 bytes 2026-02-19T20:27:06.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2245434368 bytes 2026-02-19T20:27:07.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2249719808 bytes 2026-02-19T20:27:07.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2295809024 bytes 2026-02-19T20:27:07.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2322700288 bytes 2026-02-19T20:27:07.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2344292352 bytes 2026-02-19T20:27:07.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2420695040 bytes 2026-02-19T20:27:07.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2475205632 bytes 2026-02-19T20:27:07.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2531466240 bytes 2026-02-19T20:27:07.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2568024064 bytes 2026-02-19T20:27:07.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2595261440 bytes 2026-02-19T20:27:07.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2659421184 bytes 2026-02-19T20:27:07.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2723658752 bytes 2026-02-19T20:27:07.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2768600064 bytes 2026-02-19T20:27:07.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2831245312 bytes 2026-02-19T20:27:07.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2855999488 bytes 2026-02-19T20:27:07.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2913866752 bytes 2026-02-19T20:27:07.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 2972293120 bytes 2026-02-19T20:27:08.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 2984049664 bytes 2026-02-19T20:27:08.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3028154368 bytes 2026-02-19T20:27:08.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3053825024 bytes 2026-02-19T20:27:08.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3142207488 bytes 2026-02-19T20:27:08.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3145297920 bytes 2026-02-19T20:27:08.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3162266624 bytes 2026-02-19T20:27:08.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3184364544 bytes 2026-02-19T20:27:08.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3227796480 bytes 2026-02-19T20:27:08.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3268327424 bytes 2026-02-19T20:27:08.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3314955264 bytes 2026-02-19T20:27:08.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3315562496 bytes 2026-02-19T20:27:08.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3346198528 bytes 2026-02-19T20:27:08.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3418599424 bytes 2026-02-19T20:27:08.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3461934080 bytes 2026-02-19T20:27:08.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3507996672 bytes 2026-02-19T20:27:08.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3533113344 bytes 2026-02-19T20:27:08.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3578357760 bytes 2026-02-19T20:27:08.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3584903168 bytes 2026-02-19T20:27:08.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3589344256 bytes 2026-02-19T20:27:09.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3680918528 bytes 2026-02-19T20:27:09.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3718848512 bytes 2026-02-19T20:27:09.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3758310400 bytes 2026-02-19T20:27:09.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3811459072 bytes 2026-02-19T20:27:09.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3870832640 bytes 2026-02-19T20:27:09.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3994157056 bytes 2026-02-19T20:27:09.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4034936832 bytes 2026-02-19T20:27:09.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4067720192 bytes 2026-02-19T20:27:09.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4120199168 bytes 2026-02-19T20:27:09.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4176359424 bytes 2026-02-19T20:27:09.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4209165312 bytes 2026-02-19T20:27:09.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4216342528 bytes 2026-02-19T20:27:09.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4217933824 bytes 2026-02-19T20:27:09.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4238028800 bytes 2026-02-19T20:27:09.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4299428864 bytes 2026-02-19T20:27:09.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4374415360 bytes 2026-02-19T20:27:10.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4402831360 bytes 2026-02-19T20:27:10.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4478048256 bytes 2026-02-19T20:27:10.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4557631488 bytes 2026-02-19T20:27:10.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4650562560 bytes 2026-02-19T20:27:10.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4663119872 bytes 2026-02-19T20:27:10.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4702022656 bytes 2026-02-19T20:27:10.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4759746560 bytes 2026-02-19T20:27:10.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4797599744 bytes 2026-02-19T20:27:10.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4837001216 bytes 2026-02-19T20:27:10.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4902749184 bytes 2026-02-19T20:27:10.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4931877888 bytes 2026-02-19T20:27:10.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4996316160 bytes 2026-02-19T20:27:10.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5044943872 bytes 2026-02-19T20:27:10.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5109287936 bytes 2026-02-19T20:27:11.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5171534848 bytes 2026-02-19T20:27:11.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5172130816 bytes 2026-02-19T20:27:11.095 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.29457655418910644) > weight (0.25) 2026-02-19T20:27:11.095 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6075286247801082) > weight (0.25) 2026-02-19T20:27:11.095 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4765947438179208) > weight (0.25) 2026-02-19T20:27:11.095 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5587995710034716) > weight (0.25) 2026-02-19T20:27:11.095 INFO:tasks.rbd_mirror_thrash:waiting for 102.06414562025314 secs before thrashing 2026-02-19T20:27:11.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5231364096 bytes 2026-02-19T20:27:11.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5341308928 bytes 2026-02-19T20:27:11.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5385138176 bytes 2026-02-19T20:27:11.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5411716096 bytes 2026-02-19T20:27:11.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5466617856 bytes 2026-02-19T20:27:11.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5524534272 bytes 2026-02-19T20:27:11.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5526477824 bytes 2026-02-19T20:27:11.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5553031168 bytes 2026-02-19T20:27:11.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5705861120 bytes 2026-02-19T20:27:11.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5750891520 bytes 2026-02-19T20:27:11.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5770268672 bytes 2026-02-19T20:27:11.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5780061184 bytes 2026-02-19T20:27:11.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5795869696 bytes 2026-02-19T20:27:11.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5840090112 bytes 2026-02-19T20:27:12.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5922872320 bytes 2026-02-19T20:27:12.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6001093632 bytes 2026-02-19T20:27:12.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6026774528 bytes 2026-02-19T20:27:12.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6076997632 bytes 2026-02-19T20:27:12.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6106764288 bytes 2026-02-19T20:27:12.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6129585152 bytes 2026-02-19T20:27:12.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6175647744 bytes 2026-02-19T20:27:12.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6208336896 bytes 2026-02-19T20:27:12.518 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6255448064 bytes 2026-02-19T20:27:12.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6318953472 bytes 2026-02-19T20:27:12.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6356088832 bytes 2026-02-19T20:27:12.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6363791360 bytes 2026-02-19T20:27:12.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6405914624 bytes 2026-02-19T20:27:12.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6473867264 bytes 2026-02-19T20:27:12.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6546847744 bytes 2026-02-19T20:27:12.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6573236224 bytes 2026-02-19T20:27:13.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6600633344 bytes 2026-02-19T20:27:13.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6628572160 bytes 2026-02-19T20:27:13.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6629161984 bytes 2026-02-19T20:27:13.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6674332672 bytes 2026-02-19T20:27:13.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6709142528 bytes 2026-02-19T20:27:13.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6751902720 bytes 2026-02-19T20:27:13.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6801804288 bytes 2026-02-19T20:27:13.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6824815616 bytes 2026-02-19T20:27:13.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6862418944 bytes 2026-02-19T20:27:13.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6893450240 bytes 2026-02-19T20:27:13.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6921160704 bytes 2026-02-19T20:27:13.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6946665472 bytes 2026-02-19T20:27:13.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7017800704 bytes 2026-02-19T20:27:13.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7066915840 bytes 2026-02-19T20:27:13.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7102960640 bytes 2026-02-19T20:27:13.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7130181632 bytes 2026-02-19T20:27:13.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7137589248 bytes 2026-02-19T20:27:13.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7228108800 bytes 2026-02-19T20:27:14.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7270503424 bytes 2026-02-19T20:27:14.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7271110656 bytes 2026-02-19T20:27:14.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7338334208 bytes 2026-02-19T20:27:14.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7386046464 bytes 2026-02-19T20:27:14.292 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7445572608 bytes 2026-02-19T20:27:14.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7449072640 bytes 2026-02-19T20:27:14.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7463709696 bytes 2026-02-19T20:27:14.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7490045952 bytes 2026-02-19T20:27:14.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7527776256 bytes 2026-02-19T20:27:14.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7540985856 bytes 2026-02-19T20:27:14.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7606818816 bytes 2026-02-19T20:27:14.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7630391296 bytes 2026-02-19T20:27:14.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7652458496 bytes 2026-02-19T20:27:14.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7685069824 bytes 2026-02-19T20:27:14.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7754971136 bytes 2026-02-19T20:27:14.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7770528768 bytes 2026-02-19T20:27:14.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7843098624 bytes 2026-02-19T20:27:15.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7924213760 bytes 2026-02-19T20:27:15.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7941548032 bytes 2026-02-19T20:27:15.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7984492544 bytes 2026-02-19T20:27:15.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8010001408 bytes 2026-02-19T20:27:15.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8032913408 bytes 2026-02-19T20:27:15.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8054838272 bytes 2026-02-19T20:27:15.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8124361728 bytes 2026-02-19T20:27:15.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8168376320 bytes 2026-02-19T20:27:15.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8219645952 bytes 2026-02-19T20:27:15.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8271516672 bytes 2026-02-19T20:27:15.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8329150464 bytes 2026-02-19T20:27:15.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:27:15.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:27:15.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:27:15.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:15.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:27:15.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_49 2026-02-19T20:27:15.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_49 snapshot 128M 2026-02-19T20:27:15.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:15.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_49 128M 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:15.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:15.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:27:15.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:15.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:27:15.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:27:15.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:27:15.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:15.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_49 2026-02-19T20:27:15.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:27:15.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:15.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:27:15.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:15.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_49 snapshot 2026-02-19T20:27:15.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:15.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:15.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:27:15.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:27:15.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_49 snapshot 2026-02-19T20:27:16.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:27:16.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_49 2026-02-19T20:27:16.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_49': 2026-02-19T20:27:16.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:27:16.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:27:16.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:27:16.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 14cf798fd867 2026-02-19T20:27:16.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.14cf798fd867 2026-02-19T20:27:16.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:27:16.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:27:16.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:27:16.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:27:16.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:27:15 2026 2026-02-19T20:27:16.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:27:15 2026 2026-02-19T20:27:16.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:27:15 2026 2026-02-19T20:27:16.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:27:16.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:27:16.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 6b13aa42-62d2-4bf0-b934-8f6d6bb3c8da 2026-02-19T20:27:16.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:27:16.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:27:16.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_49 2026-02-19T20:27:16.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:16.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:16.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:27:16.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:27:16.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:27:16.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:27:16.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_49 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:27:16.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 43898880 bytes 2026-02-19T20:27:16.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 113159168 bytes 2026-02-19T20:27:16.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 139021312 bytes 2026-02-19T20:27:16.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 162463744 bytes 2026-02-19T20:27:16.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 204828672 bytes 2026-02-19T20:27:17.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 268220416 bytes 2026-02-19T20:27:17.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 351765504 bytes 2026-02-19T20:27:17.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 435226624 bytes 2026-02-19T20:27:17.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 490532864 bytes 2026-02-19T20:27:17.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 518162432 bytes 2026-02-19T20:27:17.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 538387456 bytes 2026-02-19T20:27:17.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 564451328 bytes 2026-02-19T20:27:17.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 572776448 bytes 2026-02-19T20:27:17.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 607937536 bytes 2026-02-19T20:27:17.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 674950144 bytes 2026-02-19T20:27:17.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 683632640 bytes 2026-02-19T20:27:17.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 711685120 bytes 2026-02-19T20:27:17.762 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 775171072 bytes 2026-02-19T20:27:17.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 843021312 bytes 2026-02-19T20:27:17.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 865618944 bytes 2026-02-19T20:27:17.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 910962688 bytes 2026-02-19T20:27:18.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 972238848 bytes 2026-02-19T20:27:18.074 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1018181632 bytes 2026-02-19T20:27:18.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1056857088 bytes 2026-02-19T20:27:18.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1057462272 bytes 2026-02-19T20:27:18.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1119847424 bytes 2026-02-19T20:27:18.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1142825984 bytes 2026-02-19T20:27:18.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1145746432 bytes 2026-02-19T20:27:18.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1219009536 bytes 2026-02-19T20:27:18.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1307991040 bytes 2026-02-19T20:27:18.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1355600896 bytes 2026-02-19T20:27:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1356441600 bytes 2026-02-19T20:27:18.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1383674880 bytes 2026-02-19T20:27:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1398456320 bytes 2026-02-19T20:27:18.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1409213440 bytes 2026-02-19T20:27:18.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1506061312 bytes 2026-02-19T20:27:18.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1627043840 bytes 2026-02-19T20:27:18.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1631034368 bytes 2026-02-19T20:27:19.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1708737536 bytes 2026-02-19T20:27:19.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1719549952 bytes 2026-02-19T20:27:19.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1748508672 bytes 2026-02-19T20:27:19.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1797354496 bytes 2026-02-19T20:27:19.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1888649216 bytes 2026-02-19T20:27:19.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1976095744 bytes 2026-02-19T20:27:19.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2043073536 bytes 2026-02-19T20:27:19.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2106316800 bytes 2026-02-19T20:27:19.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2146505728 bytes 2026-02-19T20:27:19.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2237988864 bytes 2026-02-19T20:27:19.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2294950912 bytes 2026-02-19T20:27:19.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2337972224 bytes 2026-02-19T20:27:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2373667840 bytes 2026-02-19T20:27:19.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2421387264 bytes 2026-02-19T20:27:20.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2480291840 bytes 2026-02-19T20:27:20.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2501910528 bytes 2026-02-19T20:27:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2597753856 bytes 2026-02-19T20:27:20.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2621662208 bytes 2026-02-19T20:27:20.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2670068736 bytes 2026-02-19T20:27:20.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2726010880 bytes 2026-02-19T20:27:20.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2744773632 bytes 2026-02-19T20:27:20.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2769800192 bytes 2026-02-19T20:27:20.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2818915328 bytes 2026-02-19T20:27:20.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2847196160 bytes 2026-02-19T20:27:20.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2888088576 bytes 2026-02-19T20:27:20.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2938034176 bytes 2026-02-19T20:27:20.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2955468800 bytes 2026-02-19T20:27:20.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2982336512 bytes 2026-02-19T20:27:20.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3084886016 bytes 2026-02-19T20:27:20.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3096720384 bytes 2026-02-19T20:27:21.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3135476736 bytes 2026-02-19T20:27:21.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3152835584 bytes 2026-02-19T20:27:21.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3179515904 bytes 2026-02-19T20:27:21.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3196137472 bytes 2026-02-19T20:27:21.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3205471232 bytes 2026-02-19T20:27:21.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3275320320 bytes 2026-02-19T20:27:21.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3310637056 bytes 2026-02-19T20:27:21.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3379073024 bytes 2026-02-19T20:27:21.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3432253440 bytes 2026-02-19T20:27:21.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3464258560 bytes 2026-02-19T20:27:21.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3482973184 bytes 2026-02-19T20:27:21.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3527993344 bytes 2026-02-19T20:27:21.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3528619008 bytes 2026-02-19T20:27:21.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3609406464 bytes 2026-02-19T20:27:21.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3610028032 bytes 2026-02-19T20:27:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3666958336 bytes 2026-02-19T20:27:21.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3702642688 bytes 2026-02-19T20:27:21.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3725427712 bytes 2026-02-19T20:27:21.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3728824320 bytes 2026-02-19T20:27:22.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3755540480 bytes 2026-02-19T20:27:22.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3764935680 bytes 2026-02-19T20:27:22.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3793377280 bytes 2026-02-19T20:27:22.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3812205568 bytes 2026-02-19T20:27:22.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3863586816 bytes 2026-02-19T20:27:22.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3913231360 bytes 2026-02-19T20:27:22.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3983789056 bytes 2026-02-19T20:27:22.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4030592000 bytes 2026-02-19T20:27:22.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4084539392 bytes 2026-02-19T20:27:22.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4118064128 bytes 2026-02-19T20:27:22.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4192633856 bytes 2026-02-19T20:27:22.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4255218688 bytes 2026-02-19T20:27:22.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4282366976 bytes 2026-02-19T20:27:22.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4307919872 bytes 2026-02-19T20:27:22.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4333410304 bytes 2026-02-19T20:27:22.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4389295104 bytes 2026-02-19T20:27:23.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4474004480 bytes 2026-02-19T20:27:23.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4522676224 bytes 2026-02-19T20:27:23.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4576913408 bytes 2026-02-19T20:27:23.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4599760896 bytes 2026-02-19T20:27:23.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4600383488 bytes 2026-02-19T20:27:23.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4606427136 bytes 2026-02-19T20:27:23.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4665864192 bytes 2026-02-19T20:27:23.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4745683968 bytes 2026-02-19T20:27:23.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4794518528 bytes 2026-02-19T20:27:23.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4820621312 bytes 2026-02-19T20:27:23.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4894428160 bytes 2026-02-19T20:27:23.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4895978496 bytes 2026-02-19T20:27:23.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4954647552 bytes 2026-02-19T20:27:24.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5009639424 bytes 2026-02-19T20:27:24.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5010977792 bytes 2026-02-19T20:27:24.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5036583936 bytes 2026-02-19T20:27:24.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5068407808 bytes 2026-02-19T20:27:24.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5186600960 bytes 2026-02-19T20:27:24.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5232368640 bytes 2026-02-19T20:27:24.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5280665600 bytes 2026-02-19T20:27:24.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5314523136 bytes 2026-02-19T20:27:24.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5349968896 bytes 2026-02-19T20:27:24.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5381936128 bytes 2026-02-19T20:27:24.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5481536512 bytes 2026-02-19T20:27:24.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5503732736 bytes 2026-02-19T20:27:24.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5540668416 bytes 2026-02-19T20:27:24.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5546607616 bytes 2026-02-19T20:27:24.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5579489280 bytes 2026-02-19T20:27:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5604431872 bytes 2026-02-19T20:27:24.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5630136320 bytes 2026-02-19T20:27:25.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5718070272 bytes 2026-02-19T20:27:25.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5745501184 bytes 2026-02-19T20:27:25.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5779449856 bytes 2026-02-19T20:27:25.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5834457088 bytes 2026-02-19T20:27:25.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5858001920 bytes 2026-02-19T20:27:25.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5926158336 bytes 2026-02-19T20:27:25.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5945383936 bytes 2026-02-19T20:27:25.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5989726208 bytes 2026-02-19T20:27:25.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6074688512 bytes 2026-02-19T20:27:25.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6138499072 bytes 2026-02-19T20:27:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6200844288 bytes 2026-02-19T20:27:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6230439936 bytes 2026-02-19T20:27:25.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6283112448 bytes 2026-02-19T20:27:25.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6310862848 bytes 2026-02-19T20:27:25.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6315449344 bytes 2026-02-19T20:27:25.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6356427776 bytes 2026-02-19T20:27:26.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6420268032 bytes 2026-02-19T20:27:26.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6552728576 bytes 2026-02-19T20:27:26.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6561082368 bytes 2026-02-19T20:27:26.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6672561152 bytes 2026-02-19T20:27:26.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6673164288 bytes 2026-02-19T20:27:26.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6721645568 bytes 2026-02-19T20:27:26.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6793915392 bytes 2026-02-19T20:27:26.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6833487872 bytes 2026-02-19T20:27:26.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6909341696 bytes 2026-02-19T20:27:26.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6982372352 bytes 2026-02-19T20:27:26.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7115879424 bytes 2026-02-19T20:27:26.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7159074816 bytes 2026-02-19T20:27:26.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7206949888 bytes 2026-02-19T20:27:26.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7243035648 bytes 2026-02-19T20:27:27.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7260929024 bytes 2026-02-19T20:27:27.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7325526016 bytes 2026-02-19T20:27:27.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7335871488 bytes 2026-02-19T20:27:27.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7364879360 bytes 2026-02-19T20:27:27.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7384335360 bytes 2026-02-19T20:27:27.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7404374016 bytes 2026-02-19T20:27:27.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7451905024 bytes 2026-02-19T20:27:27.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7529375744 bytes 2026-02-19T20:27:27.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7545337856 bytes 2026-02-19T20:27:27.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7592150016 bytes 2026-02-19T20:27:27.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7696317440 bytes 2026-02-19T20:27:27.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7736724480 bytes 2026-02-19T20:27:27.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7771675648 bytes 2026-02-19T20:27:27.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7807472640 bytes 2026-02-19T20:27:27.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7861565440 bytes 2026-02-19T20:27:27.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7887830016 bytes 2026-02-19T20:27:28.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7963071488 bytes 2026-02-19T20:27:28.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 8031397888 bytes 2026-02-19T20:27:28.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 8062577664 bytes 2026-02-19T20:27:28.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 8130263040 bytes 2026-02-19T20:27:28.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 8260634624 bytes 2026-02-19T20:27:28.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 8284803072 bytes 2026-02-19T20:27:28.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8315439104 bytes 2026-02-19T20:27:28.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8364666880 bytes 2026-02-19T20:27:28.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8404564992 bytes 2026-02-19T20:27:28.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:27:28.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:27:28.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:27:28.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:28.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_50 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image_and_enable_mirror cluster2 mirror image_50 snapshot 128M 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n snapshot ']' 2026-02-19T20:27:28.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_50 128M 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 128M ']' 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=128M 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:27:28.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering mirror/image_50 2026-02-19T20:27:28.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ is_pool_mirror_mode_image mirror 2026-02-19T20:27:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' image = image ']' 2026-02-19T20:27:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_50 snapshot 2026-02-19T20:27:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:27:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:27:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_50 snapshot 2026-02-19T20:27:29.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:27:29.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_50 2026-02-19T20:27:29.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_50': 2026-02-19T20:27:29.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 14dbc0eb6532 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.14dbc0eb6532 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:27:28 2026 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:27:28 2026 2026-02-19T20:27:29.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:27:28 2026 2026-02-19T20:27:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:27:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:27:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 5f2c96e0-e5f7-4818-9461-1ddee61be14f 2026-02-19T20:27:29.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:27:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n '' ']' 2026-02-19T20:27:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_50 2026-02-19T20:27:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:27:29.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:27:29.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=9 2026-02-19T20:27:29.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:27:29.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_50 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:27:29.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 91019264 bytes 2026-02-19T20:27:29.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 136302592 bytes 2026-02-19T20:27:29.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 210914304 bytes 2026-02-19T20:27:29.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 289162240 bytes 2026-02-19T20:27:30.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 335361024 bytes 2026-02-19T20:27:30.101 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 368621568 bytes 2026-02-19T20:27:30.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 416175104 bytes 2026-02-19T20:27:30.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 420293632 bytes 2026-02-19T20:27:30.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 462584832 bytes 2026-02-19T20:27:30.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 504811520 bytes 2026-02-19T20:27:30.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 547270656 bytes 2026-02-19T20:27:30.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 600777728 bytes 2026-02-19T20:27:30.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 641943552 bytes 2026-02-19T20:27:30.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 683038720 bytes 2026-02-19T20:27:30.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 718643200 bytes 2026-02-19T20:27:30.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 785960960 bytes 2026-02-19T20:27:30.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 803966976 bytes 2026-02-19T20:27:30.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 860142592 bytes 2026-02-19T20:27:30.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 889277440 bytes 2026-02-19T20:27:30.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 943414272 bytes 2026-02-19T20:27:31.083 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1052540928 bytes 2026-02-19T20:27:31.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1070164992 bytes 2026-02-19T20:27:31.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1102750720 bytes 2026-02-19T20:27:31.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1148550144 bytes 2026-02-19T20:27:31.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1202642944 bytes 2026-02-19T20:27:31.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1227752448 bytes 2026-02-19T20:27:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1274346496 bytes 2026-02-19T20:27:31.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1313497088 bytes 2026-02-19T20:27:31.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1368847360 bytes 2026-02-19T20:27:31.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1369968640 bytes 2026-02-19T20:27:31.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1433943040 bytes 2026-02-19T20:27:31.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1468034048 bytes 2026-02-19T20:27:31.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1523553280 bytes 2026-02-19T20:27:31.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1592944640 bytes 2026-02-19T20:27:31.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1655689216 bytes 2026-02-19T20:27:32.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1748084736 bytes 2026-02-19T20:27:32.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1823094784 bytes 2026-02-19T20:27:32.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1848471552 bytes 2026-02-19T20:27:32.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1867122688 bytes 2026-02-19T20:27:32.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1868920832 bytes 2026-02-19T20:27:32.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1905171456 bytes 2026-02-19T20:27:32.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1951522816 bytes 2026-02-19T20:27:32.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2036328448 bytes 2026-02-19T20:27:32.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2075488256 bytes 2026-02-19T20:27:32.579 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2091827200 bytes 2026-02-19T20:27:32.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2165340160 bytes 2026-02-19T20:27:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2292198400 bytes 2026-02-19T20:27:32.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2369705984 bytes 2026-02-19T20:27:32.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2425650176 bytes 2026-02-19T20:27:33.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2476845056 bytes 2026-02-19T20:27:33.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2535200768 bytes 2026-02-19T20:27:33.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2591644672 bytes 2026-02-19T20:27:33.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2623927296 bytes 2026-02-19T20:27:33.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2663271424 bytes 2026-02-19T20:27:33.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2694903808 bytes 2026-02-19T20:27:33.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2730180608 bytes 2026-02-19T20:27:33.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2778481664 bytes 2026-02-19T20:27:33.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2832861184 bytes 2026-02-19T20:27:33.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2859881472 bytes 2026-02-19T20:27:33.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2919686144 bytes 2026-02-19T20:27:33.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2950309888 bytes 2026-02-19T20:27:33.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2995536896 bytes 2026-02-19T20:27:33.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 3029228544 bytes 2026-02-19T20:27:33.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 3064031232 bytes 2026-02-19T20:27:33.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3083531264 bytes 2026-02-19T20:27:34.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3138008064 bytes 2026-02-19T20:27:34.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3214604288 bytes 2026-02-19T20:27:34.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3215217664 bytes 2026-02-19T20:27:34.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3244503040 bytes 2026-02-19T20:27:34.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3246639104 bytes 2026-02-19T20:27:34.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3316253696 bytes 2026-02-19T20:27:34.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3316896768 bytes 2026-02-19T20:27:34.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3353623552 bytes 2026-02-19T20:27:34.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3439774720 bytes 2026-02-19T20:27:34.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3509193728 bytes 2026-02-19T20:27:34.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3557660672 bytes 2026-02-19T20:27:34.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3596265472 bytes 2026-02-19T20:27:34.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3618787328 bytes 2026-02-19T20:27:34.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3629120512 bytes 2026-02-19T20:27:34.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3681588224 bytes 2026-02-19T20:27:34.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3702263808 bytes 2026-02-19T20:27:34.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3795347456 bytes 2026-02-19T20:27:35.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3810237440 bytes 2026-02-19T20:27:35.077 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3873355776 bytes 2026-02-19T20:27:35.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3896952832 bytes 2026-02-19T20:27:35.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3897968640 bytes 2026-02-19T20:27:35.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3956315136 bytes 2026-02-19T20:27:35.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 4016812032 bytes 2026-02-19T20:27:35.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4049378304 bytes 2026-02-19T20:27:35.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4050781184 bytes 2026-02-19T20:27:35.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4083015680 bytes 2026-02-19T20:27:35.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4141533184 bytes 2026-02-19T20:27:35.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4183721984 bytes 2026-02-19T20:27:35.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4215353344 bytes 2026-02-19T20:27:35.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4246766592 bytes 2026-02-19T20:27:35.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4299293696 bytes 2026-02-19T20:27:35.889 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4394348544 bytes 2026-02-19T20:27:35.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4440546304 bytes 2026-02-19T20:27:36.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4487574528 bytes 2026-02-19T20:27:36.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4529869824 bytes 2026-02-19T20:27:36.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4579133440 bytes 2026-02-19T20:27:36.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4582760448 bytes 2026-02-19T20:27:36.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4643514368 bytes 2026-02-19T20:27:36.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4682157056 bytes 2026-02-19T20:27:36.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4705530880 bytes 2026-02-19T20:27:36.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4732307456 bytes 2026-02-19T20:27:36.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4774147072 bytes 2026-02-19T20:27:36.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4855723008 bytes 2026-02-19T20:27:36.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4923031552 bytes 2026-02-19T20:27:36.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4969723904 bytes 2026-02-19T20:27:36.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5001437184 bytes 2026-02-19T20:27:36.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5027089408 bytes 2026-02-19T20:27:36.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5079152640 bytes 2026-02-19T20:27:36.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5114271744 bytes 2026-02-19T20:27:37.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5150332928 bytes 2026-02-19T20:27:37.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5205830656 bytes 2026-02-19T20:27:37.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5237565440 bytes 2026-02-19T20:27:37.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5238187008 bytes 2026-02-19T20:27:37.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5247252480 bytes 2026-02-19T20:27:37.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5288798208 bytes 2026-02-19T20:27:37.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5322399744 bytes 2026-02-19T20:27:37.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5351506944 bytes 2026-02-19T20:27:37.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5447906304 bytes 2026-02-19T20:27:37.539 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5468537856 bytes 2026-02-19T20:27:37.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5506829312 bytes 2026-02-19T20:27:37.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5507445760 bytes 2026-02-19T20:27:37.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5526827008 bytes 2026-02-19T20:27:37.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5545560064 bytes 2026-02-19T20:27:37.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5572705280 bytes 2026-02-19T20:27:37.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5623443456 bytes 2026-02-19T20:27:37.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5661030400 bytes 2026-02-19T20:27:37.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5684707328 bytes 2026-02-19T20:27:37.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5689713664 bytes 2026-02-19T20:27:38.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5741794304 bytes 2026-02-19T20:27:38.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5754180608 bytes 2026-02-19T20:27:38.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5758395392 bytes 2026-02-19T20:27:38.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5822962688 bytes 2026-02-19T20:27:38.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5906294784 bytes 2026-02-19T20:27:38.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5994000384 bytes 2026-02-19T20:27:38.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6046380032 bytes 2026-02-19T20:27:38.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6110180352 bytes 2026-02-19T20:27:38.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6128525312 bytes 2026-02-19T20:27:38.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:27:38.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:27:38.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:27:38.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:38.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_pool_images cluster2 mirror 50 2026-02-19T20:27:38.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:38.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:38.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_count=50 2026-02-19T20:27:38.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:27:38.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local count 2026-02-19T20:27:38.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_count=0 2026-02-19T20:27:38.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:27:38.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 40 2026-02-19T20:27:38.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in `seq 1 40` 2026-02-19T20:27:38.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 1 -ne 1 2026-02-19T20:27:38.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster2 -p mirror mirror pool status 2026-02-19T20:27:38.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep 'images: ' 2026-02-19T20:27:38.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cut '-d ' -f 2 2026-02-19T20:27:38.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: failed to query services: (13) Permission denied 2026-02-19T20:27:38.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ count=50 2026-02-19T20:27:38.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 50 = 50 2026-02-19T20:27:38.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:38.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_pool_healthy cluster2 mirror 2026-02-19T20:27:38.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:27:38.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:38.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:27:38.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state 2026-02-19T20:27:38.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 40 2026-02-19T20:27:38.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in `seq 1 40` 2026-02-19T20:27:38.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 1 -ne 1 2026-02-19T20:27:38.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster2 -p mirror mirror pool status 2026-02-19T20:27:38.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep 'image health:' 2026-02-19T20:27:38.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cut '-d ' -f 3 2026-02-19T20:27:38.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: failed to query services: (13) Permission denied 2026-02-19T20:27:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=OK 2026-02-19T20:27:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test OK = ERROR 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test OK = OK 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_pool_images cluster1 mirror 50 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_count=50 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local count 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_count=0 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:27:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 40 2026-02-19T20:27:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in `seq 1 40` 2026-02-19T20:27:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 1 -ne 1 2026-02-19T20:27:38.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 -p mirror mirror pool status 2026-02-19T20:27:38.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep 'images: ' 2026-02-19T20:27:38.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cut '-d ' -f 2 2026-02-19T20:27:38.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: failed to query services: (13) Permission denied 2026-02-19T20:27:38.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ count=50 2026-02-19T20:27:38.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 50 = 50 2026-02-19T20:27:38.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:27:38.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_pool_healthy cluster1 mirror 2026-02-19T20:27:38.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:27:38.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:27:38.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:27:38.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state 2026-02-19T20:27:38.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 40 2026-02-19T20:27:38.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in `seq 1 40` 2026-02-19T20:27:38.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 1 -ne 1 2026-02-19T20:27:38.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 -p mirror mirror pool status 2026-02-19T20:27:38.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep 'image health:' 2026-02-19T20:27:38.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cut '-d ' -f 3 2026-02-19T20:27:38.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: failed to query services: (13) Permission denied 2026-02-19T20:27:38.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=WARNING 2026-02-19T20:27:38.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test WARNING = ERROR 2026-02-19T20:27:38.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test WARNING = OK 2026-02-19T20:27:38.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in `seq 1 40` 2026-02-19T20:27:38.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 2 -ne 1 2026-02-19T20:27:38.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 30 2026-02-19T20:28:08.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 -p mirror mirror pool status 2026-02-19T20:28:08.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep 'image health:' 2026-02-19T20:28:08.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cut '-d ' -f 3 2026-02-19T20:28:08.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: failed to query services: (13) Permission denied 2026-02-19T20:28:08.762 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=OK 2026-02-19T20:28:08.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test OK = ERROR 2026-02-19T20:28:08.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test OK = OK 2026-02-19T20:28:08.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:08.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ testlog 'TEST: compare many images' 2026-02-19T20:28:08.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ tee -a /tmp/tmp.9BL5XQhnTi/rbd-mirror.test.log 2026-02-19T20:28:08.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ date '+%F %T' 2026-02-19T20:28:08.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo 2026-02-19 20:28:08 TEST: compare many images 2026-02-19T20:28:08.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:2026-02-19 20:28:08 TEST: compare many images 2026-02-19T20:28:08.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 50 2026-02-19T20:28:08.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:28:08.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_1 2026-02-19T20:28:08.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_1 snap 2026-02-19T20:28:08.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:08.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:08.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:08.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:08.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_1@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:28:09.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_1 2026-02-19T20:28:09.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:09.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:09.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:09.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_1 started 2026-02-19T20:28:09.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:09.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:09.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:09.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:28:09.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:09.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:09.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:10.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_1 started 2026-02-19T20:28:10.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:10.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:10.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:10.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:28:10.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_1 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:10.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:28:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_1 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:10.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:28:10.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:28:10.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:28:10.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:10.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:10.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:10.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_1 2026-02-19T20:28:10.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:28:10.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:28:10.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:28:10.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:28:10.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:28:10.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:28:10.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:28:10.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_1 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_1 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:10.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:10.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_1 2026-02-19T20:28:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 86 2026-02-19T20:28:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_1 2026-02-19T20:28:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_1.status 2026-02-19T20:28:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_1.status 2026-02-19T20:28:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:11.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_1.status 2026-02-19T20:28:11.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_1.status 2026-02-19T20:28:11.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_1.status 2026-02-19T20:28:11.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:11.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.status 2026-02-19T20:28:11.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_1 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_1.status 2026-02-19T20:28:11.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:11.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:11.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:11.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_1.status 2026-02-19T20:28:11.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_1 --format xml 2026-02-19T20:28:11.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:11.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:28:11.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=86 2026-02-19T20:28:11.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:11.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:11.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:28:11.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_1 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.status 2026-02-19T20:28:11.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:11.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:11.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:11.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.status 2026-02-19T20:28:11.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_1 --format xml 2026-02-19T20:28:11.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:11.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=35 2026-02-19T20:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 86 = 35 2026-02-19T20:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:11.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:28:12.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_1 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.status 2026-02-19T20:28:12.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:12.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:12.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:12.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.status 2026-02-19T20:28:12.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_1 --format xml 2026-02-19T20:28:12.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:12.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:12.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=35 2026-02-19T20:28:12.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 86 = 35 2026-02-19T20:28:12.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:12.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:28:13.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_1 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.status 2026-02-19T20:28:13.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:13.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:13.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:13.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.status 2026-02-19T20:28:13.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_1 --format xml 2026-02-19T20:28:13.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:13.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:13.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:13.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:28:13.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:28:13.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:28:13.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:28:13.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 86 = '' 2026-02-19T20:28:13.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:13.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:28:13.431 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35787603388882505) > weight (0.25) 2026-02-19T20:28:13.431 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9906950635855443) > weight (0.25) 2026-02-19T20:28:13.431 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4574792905430243) > weight (0.25) 2026-02-19T20:28:13.431 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.40142963904808504) > weight (0.25) 2026-02-19T20:28:13.431 INFO:tasks.rbd_mirror_thrash:waiting for 71.05870106523771 secs before thrashing 2026-02-19T20:28:14.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_1 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.status 2026-02-19T20:28:14.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:14.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:14.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:14.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.status 2026-02-19T20:28:14.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_1 --format xml 2026-02-19T20:28:14.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:14.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=86 2026-02-19T20:28:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 86 = 86 2026-02-19T20:28:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_1 snap 2026-02-19T20:28:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:14.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:14.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:28:14.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:15.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_1@snap 2026-02-19T20:28:15.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_1': 2026-02-19T20:28:15.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:28:15.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:28:15.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:28:15.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b899b7b8e1 2026-02-19T20:28:15.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b899b7b8e1 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:03 2026 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:03 2026 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:03 2026 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 1bb58447-6e84-49f6-86ad-1db8d6f29daf 2026-02-19T20:28:15.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:28:15.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:15.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_1 snap 2026-02-19T20:28:15.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:15.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:28:15.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:15.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:28:15.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_1.export 2026-02-19T20:28:15.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.export 2026-02-19T20:28:15.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_1.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.export 2026-02-19T20:28:15.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_1@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_1.export 2026-02-19T20:28:15.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:15.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_1@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.export 2026-02-19T20:28:16.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:16.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_1.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.export 2026-02-19T20:28:16.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_1.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_1.export 2026-02-19T20:28:16.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:16.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:28:16.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_2 2026-02-19T20:28:16.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_2 snap 2026-02-19T20:28:16.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:16.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:16.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:16.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:16.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_2@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:28:16.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_2 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_2 started 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:16.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:17.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_2 started 2026-02-19T20:28:17.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_2 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:28:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:28:17.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:28:17.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:28:17.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:17.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:28:17.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:28:17.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:28:17.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:17.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_2 2026-02-19T20:28:17.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:28:17.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:17.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:17.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:28:17.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:17.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:28:17.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:28:17.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:17.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:17.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:17.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_2 2026-02-19T20:28:17.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:28:17.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:28:17.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:28:17.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:28:17.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:28:17.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:28:17.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:28:17.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:17.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_2 2026-02-19T20:28:17.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:17.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:17.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:17.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:17.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:17.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:28:17.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:28:17.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_2 2026-02-19T20:28:17.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:17.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:17.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:17.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_2 2026-02-19T20:28:18.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 88 2026-02-19T20:28:18.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_2 2026-02-19T20:28:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_2.status 2026-02-19T20:28:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_2.status 2026-02-19T20:28:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_2.status 2026-02-19T20:28:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_2.status 2026-02-19T20:28:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_2.status 2026-02-19T20:28:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:18.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.status 2026-02-19T20:28:18.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_2 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_2.status 2026-02-19T20:28:18.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:18.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:18.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:18.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_2.status 2026-02-19T20:28:18.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_2 --format xml 2026-02-19T20:28:18.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:18.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:28:18.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=88 2026-02-19T20:28:18.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:18.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:18.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:28:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_2 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.status 2026-02-19T20:28:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.status 2026-02-19T20:28:18.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_2 --format xml 2026-02-19T20:28:18.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:18.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:18.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=36 2026-02-19T20:28:18.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 88 = 36 2026-02-19T20:28:18.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:18.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:28:19.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_2 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.status 2026-02-19T20:28:19.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:19.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:19.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:19.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.status 2026-02-19T20:28:19.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_2 --format xml 2026-02-19T20:28:19.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:19.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:19.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=36 2026-02-19T20:28:19.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 88 = 36 2026-02-19T20:28:19.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:19.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:28:20.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_2 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.status 2026-02-19T20:28:20.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:20.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:20.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:20.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.status 2026-02-19T20:28:20.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_2 --format xml 2026-02-19T20:28:20.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:20.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:20.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:20.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:28:20.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:28:20.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:28:20.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:28:20.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 88 = '' 2026-02-19T20:28:20.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:20.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:28:21.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_2 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.status 2026-02-19T20:28:21.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:21.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:21.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:21.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.status 2026-02-19T20:28:21.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_2 --format xml 2026-02-19T20:28:21.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:21.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:21.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=88 2026-02-19T20:28:21.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 88 = 88 2026-02-19T20:28:21.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:21.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_2 snap 2026-02-19T20:28:21.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:21.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:21.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:21.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:21.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:21.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:28:21.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:22.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_2@snap 2026-02-19T20:28:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_2': 2026-02-19T20:28:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113a3933f0cb 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113a3933f0cb 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:18 2026 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:18 2026 2026-02-19T20:28:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:18 2026 2026-02-19T20:28:23.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:28:23.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:28:23.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:28:23.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 26fa2d67-71a3-4278-b257-25f69a27f0fe 2026-02-19T20:28:23.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:28:23.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:23.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_2 snap 2026-02-19T20:28:23.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:23.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:28:23.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:23.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:28:23.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_2.export 2026-02-19T20:28:23.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.export 2026-02-19T20:28:23.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_2.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.export 2026-02-19T20:28:23.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_2@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_2.export 2026-02-19T20:28:23.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:23.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_2@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.export 2026-02-19T20:28:23.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_2.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.export 2026-02-19T20:28:23.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_2.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_2.export 2026-02-19T20:28:23.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:23.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:28:23.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_3 2026-02-19T20:28:23.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_3 snap 2026-02-19T20:28:23.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:23.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:23.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:23.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:23.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_3@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:28:23.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_3 2026-02-19T20:28:23.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_3 started 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:23.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_3 started 2026-02-19T20:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_3 2026-02-19T20:28:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:28:24.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:28:24.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:24.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_3 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:24.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:28:24.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:28:24.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:28:24.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:24.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:24.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:24.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_3 2026-02-19T20:28:24.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:no valid command found; 2 closest matches: 2026-02-19T20:28:24.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_37 2026-02-19T20:28:24.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_39 2026-02-19T20:28:24.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:24.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:24.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_3 2026-02-19T20:28:24.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:28:24.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:24.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:28:24.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:28:24.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:24.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:24.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:24.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_3 2026-02-19T20:28:24.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:no valid command found; 4 closest matches: 2026-02-19T20:28:24.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_31 2026-02-19T20:28:24.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_32 2026-02-19T20:28:24.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_35 2026-02-19T20:28:24.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_38 2026-02-19T20:28:24.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:24.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:24.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_3 2026-02-19T20:28:24.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:24.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:24.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:24.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:24.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:24.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:28:24.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:28:24.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:28:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_3 2026-02-19T20:28:24.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:no valid command found; 2 closest matches: 2026-02-19T20:28:24.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_33 2026-02-19T20:28:24.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_36 2026-02-19T20:28:24.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:24.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_3 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:28:24.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:24.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:24.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:28:24.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:28:24.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:24.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:24.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:28:24.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:28:24.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:28:24.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:28:24.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:28:24.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:28:24.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_3 2026-02-19T20:28:24.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:28:24.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:28:24.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:28:24.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:28:24.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:28:24.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:28:24.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:28:24.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:24.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_3 2026-02-19T20:28:24.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:24.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:24.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:24.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:24.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:24.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:28:24.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:28:24.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_3 2026-02-19T20:28:24.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:24.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:24.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:24.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_3 2026-02-19T20:28:25.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 90 2026-02-19T20:28:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_3 2026-02-19T20:28:25.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:25.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:25.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:25.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:25.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:25.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_3.status 2026-02-19T20:28:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_3.status 2026-02-19T20:28:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_3.status 2026-02-19T20:28:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_3.status 2026-02-19T20:28:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_3.status 2026-02-19T20:28:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:25.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.status 2026-02-19T20:28:25.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_3 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_3.status 2026-02-19T20:28:25.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:25.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:25.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:25.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_3.status 2026-02-19T20:28:25.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_3 --format xml 2026-02-19T20:28:25.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:25.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:28:25.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=90 2026-02-19T20:28:25.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:25.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:25.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:28:25.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_3 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.status 2026-02-19T20:28:25.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:25.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:25.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:25.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.status 2026-02-19T20:28:25.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_3 --format xml 2026-02-19T20:28:25.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:25.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:25.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=37 2026-02-19T20:28:25.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 90 = 37 2026-02-19T20:28:25.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:25.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:28:26.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_3 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.status 2026-02-19T20:28:26.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:26.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:26.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:26.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.status 2026-02-19T20:28:26.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_3 --format xml 2026-02-19T20:28:26.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:26.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:26.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=37 2026-02-19T20:28:26.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 90 = 37 2026-02-19T20:28:26.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:26.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:28:27.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_3 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.status 2026-02-19T20:28:27.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:27.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:27.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:27.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.status 2026-02-19T20:28:27.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_3 --format xml 2026-02-19T20:28:27.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:27.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:27.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:27.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:28:27.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:28:27.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:28:27.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:28:27.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 90 = '' 2026-02-19T20:28:27.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:27.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:28:28.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_3 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.status 2026-02-19T20:28:28.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:28.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:28.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:28.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.status 2026-02-19T20:28:28.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_3 --format xml 2026-02-19T20:28:28.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:28.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:28.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=90 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 90 = 90 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_3 snap 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:28:28.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:29.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_3@snap 2026-02-19T20:28:29.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_3': 2026-02-19T20:28:29.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:28:29.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:28:29.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:28:29.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11dfe58e15fd 2026-02-19T20:28:29.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11dfe58e15fd 2026-02-19T20:28:29.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:35 2026 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:35 2026 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:35 2026 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 71651990-b478-4ee5-835f-7147f7defb72 2026-02-19T20:28:29.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_3 snap 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_3.export 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.export 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_3.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.export 2026-02-19T20:28:29.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_3@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_3.export 2026-02-19T20:28:29.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:29.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_3@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.export 2026-02-19T20:28:30.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:30.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_3.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.export 2026-02-19T20:28:30.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_3.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_3.export 2026-02-19T20:28:30.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:30.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:28:30.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_4 2026-02-19T20:28:30.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_4 snap 2026-02-19T20:28:30.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:30.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:30.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:30.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:30.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_4@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:28:30.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_4 2026-02-19T20:28:30.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_4 started 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:30.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:31.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_4 started 2026-02-19T20:28:31.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:31.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:31.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:31.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:28:31.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:28:31.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:28:31.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_4 2026-02-19T20:28:31.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:28:31.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:31.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:28:31.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:28:31.262 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:28:31.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:28:31.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:31.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:28:31.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_4 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:31.266 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:28:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:31.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:31.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:31.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:31.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:31.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:31.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:31.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:28:31.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:28:31.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:28:31.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:31.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:31.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:31.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_4 2026-02-19T20:28:31.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:no valid command found; 2 closest matches: 2026-02-19T20:28:31.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_43 2026-02-19T20:28:31.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_47 2026-02-19T20:28:31.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:31.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:31.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_4 2026-02-19T20:28:31.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:28:31.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:31.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:31.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:28:31.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:28:31.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:31.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:28:31.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:28:31.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:31.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:31.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:31.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_4 2026-02-19T20:28:31.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:no valid command found; 2 closest matches: 2026-02-19T20:28:31.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_40 2026-02-19T20:28:31.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_46 2026-02-19T20:28:31.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:31.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:31.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_4 2026-02-19T20:28:31.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:28:31.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:31.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:31.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:28:31.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:31.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:28:31.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:28:31.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:28:31.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:31.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:31.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:31.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_4 2026-02-19T20:28:31.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:no valid command found; 4 closest matches: 2026-02-19T20:28:31.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_41 2026-02-19T20:28:31.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_42 2026-02-19T20:28:31.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_44 2026-02-19T20:28:31.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_48 2026-02-19T20:28:31.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_4 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:31.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:28:31.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:28:31.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:28:31.439 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:28:31.439 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:28:31.439 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:28:31.439 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_4 2026-02-19T20:28:31.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:28:31.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:28:31.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:28:31.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:28:31.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_4 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_4 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:31.490 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_4 2026-02-19T20:28:32.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 92 2026-02-19T20:28:32.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_4 2026-02-19T20:28:32.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:32.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:32.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:32.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:32.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:32.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_4.status 2026-02-19T20:28:32.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_4.status 2026-02-19T20:28:32.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:32.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_4.status 2026-02-19T20:28:32.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_4.status 2026-02-19T20:28:32.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_4.status 2026-02-19T20:28:32.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:32.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.status 2026-02-19T20:28:32.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_4 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_4.status 2026-02-19T20:28:32.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:32.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:32.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:32.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_4.status 2026-02-19T20:28:32.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_4 --format xml 2026-02-19T20:28:32.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:32.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:28:32.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=92 2026-02-19T20:28:32.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:32.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:32.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:28:32.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_4 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.status 2026-02-19T20:28:32.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:32.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:32.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:32.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.status 2026-02-19T20:28:32.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_4 --format xml 2026-02-19T20:28:32.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:32.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:32.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=38 2026-02-19T20:28:32.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 92 = 38 2026-02-19T20:28:32.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:32.532 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:28:32.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_4 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.status 2026-02-19T20:28:32.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:32.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:32.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:32.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.status 2026-02-19T20:28:32.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_4 --format xml 2026-02-19T20:28:32.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:32.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:32.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:32.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:28:32.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:28:32.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:28:32.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:28:32.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 92 = '' 2026-02-19T20:28:32.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:32.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:28:33.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_4 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.status 2026-02-19T20:28:33.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:33.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:33.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:33.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.status 2026-02-19T20:28:33.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_4 --format xml 2026-02-19T20:28:33.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:33.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:33.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=92 2026-02-19T20:28:33.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 92 = 92 2026-02-19T20:28:33.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:33.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_4 snap 2026-02-19T20:28:33.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:33.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:33.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:33.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:33.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:33.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:28:33.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:34.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_4@snap 2026-02-19T20:28:34.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_4': 2026-02-19T20:28:34.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:28:34.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123da3b1d800 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123da3b1d800 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:43 2026 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:43 2026 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:43 2026 2026-02-19T20:28:34.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:28:34.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:28:34.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:28:34.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 03107192-aafc-4953-a9c6-561e8976e03f 2026-02-19T20:28:34.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:28:34.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:34.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_4 snap 2026-02-19T20:28:34.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:34.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:28:34.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:34.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:28:34.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_4.export 2026-02-19T20:28:34.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.export 2026-02-19T20:28:34.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_4.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.export 2026-02-19T20:28:34.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_4@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_4.export 2026-02-19T20:28:34.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:34.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_4@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.export 2026-02-19T20:28:35.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:35.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_4.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.export 2026-02-19T20:28:35.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_4.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_4.export 2026-02-19T20:28:35.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:35.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:28:35.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_5 2026-02-19T20:28:35.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_5 snap 2026-02-19T20:28:35.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:35.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:35.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:35.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:35.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_5@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_5 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_5 started 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:28:35.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:35.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:35.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:36.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_5 started 2026-02-19T20:28:36.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:28:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:28:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:28:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_5 2026-02-19T20:28:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:28:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:28:36.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:36.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:28:36.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:28:36.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:28:36.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:36.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:28:36.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:28:36.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:28:36.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:36.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_5 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:36.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:36.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:36.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:36.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:36.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:36.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:28:36.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:28:36.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:28:36.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:36.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:36.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:36.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_5 2026-02-19T20:28:36.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2026-02-19T20:28:36.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:28:36.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:36.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:36.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_5 2026-02-19T20:28:36.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:28:36.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:36.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:36.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:28:36.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:28:36.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:36.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:28:36.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:28:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_5 2026-02-19T20:28:36.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:no valid command found; 1 closest matches: 2026-02-19T20:28:36.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status mirror/image_50 2026-02-19T20:28:36.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:36.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:36.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_5 2026-02-19T20:28:36.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:36.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:36.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:36.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:36.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:28:36.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:28:36.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:28:36.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:36.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:36.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:36.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_5 2026-02-19T20:28:36.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_5'] 2026-02-19T20:28:36.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:28:36.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:36.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:36.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_5 2026-02-19T20:28:36.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:28:36.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:36.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:28:36.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:36.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:28:36.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:28:36.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:28:36.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:28:36.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:28:36.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:28:36.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_5 2026-02-19T20:28:36.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:28:36.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:28:36.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:28:36.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:28:36.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:28:36.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:28:36.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:28:36.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:36.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_5 2026-02-19T20:28:36.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_5 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:36.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_5 2026-02-19T20:28:37.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 94 2026-02-19T20:28:37.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_5 2026-02-19T20:28:37.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:37.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:37.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:37.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:37.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:37.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_5.status 2026-02-19T20:28:37.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_5.status 2026-02-19T20:28:37.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:37.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_5.status 2026-02-19T20:28:37.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_5.status 2026-02-19T20:28:37.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_5.status 2026-02-19T20:28:37.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:37.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.status 2026-02-19T20:28:37.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_5 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_5.status 2026-02-19T20:28:37.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:37.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:37.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:37.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_5.status 2026-02-19T20:28:37.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_5 --format xml 2026-02-19T20:28:37.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:37.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:28:37.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=94 2026-02-19T20:28:37.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:37.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:37.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:28:37.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_5 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.status 2026-02-19T20:28:37.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:37.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:37.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:37.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.status 2026-02-19T20:28:37.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_5 --format xml 2026-02-19T20:28:37.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:37.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:37.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=39 2026-02-19T20:28:37.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 94 = 39 2026-02-19T20:28:37.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:37.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:28:38.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_5 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.status 2026-02-19T20:28:38.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:38.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:38.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:38.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.status 2026-02-19T20:28:38.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_5 --format xml 2026-02-19T20:28:38.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:38.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:38.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=39 2026-02-19T20:28:38.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 94 = 39 2026-02-19T20:28:38.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:38.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:28:39.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_5 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.status 2026-02-19T20:28:39.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:39.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:39.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:39.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.status 2026-02-19T20:28:39.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_5 --format xml 2026-02-19T20:28:39.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:39.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:39.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:39.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:28:39.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:28:39.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:28:39.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:28:39.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 94 = '' 2026-02-19T20:28:39.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:39.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:28:40.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_5 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.status 2026-02-19T20:28:40.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:40.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:40.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:40.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.status 2026-02-19T20:28:40.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_5 --format xml 2026-02-19T20:28:40.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:40.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:40.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=94 2026-02-19T20:28:40.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 94 = 94 2026-02-19T20:28:40.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:40.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_5 snap 2026-02-19T20:28:40.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:40.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:40.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:40.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:40.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:40.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:28:40.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:41.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_5@snap 2026-02-19T20:28:41.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_5': 2026-02-19T20:28:41.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:28:41.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:28:41.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:28:41.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b83e7ec366 2026-02-19T20:28:41.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b83e7ec366 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:18:57 2026 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:18:57 2026 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:18:57 2026 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: e1d197e7-fd23-4005-8baf-4a772b01d8ee 2026-02-19T20:28:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_5 snap 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_5.export 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.export 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_5.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.export 2026-02-19T20:28:41.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_5@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_5.export 2026-02-19T20:28:41.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:41.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_5@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.export 2026-02-19T20:28:42.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:42.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_5.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.export 2026-02-19T20:28:42.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_5.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_5.export 2026-02-19T20:28:42.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:42.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:28:42.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_6 2026-02-19T20:28:42.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_6 snap 2026-02-19T20:28:42.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:42.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:42.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:42.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:42.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_6@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:28:42.591 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:28:42.588+0000 7fab1dc3d640 -1 reset not still connected to 0x5f2d380 2026-02-19T20:28:42.592 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:28:42.589+0000 7fab1dc3d640 -1 reset not still connected to 0x651edd0 2026-02-19T20:28:42.592 INFO:tasks.ceph.cluster2.osd.2.trial080.stderr:2026-02-19T20:28:42.589+0000 7fab1dc3d640 -1 reset not still connected to 0x28f27040 2026-02-19T20:28:42.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_6 2026-02-19T20:28:42.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:42.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:42.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:42.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_6 started 2026-02-19T20:28:42.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:42.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:42.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:42.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:28:42.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:42.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:42.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:43.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_6 started 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_6 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:43.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:28:43.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:28:43.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:28:43.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:43.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:28:43.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:28:43.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:28:43.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:28:43.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:43.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_6 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:43.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:43.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:43.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:43.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:43.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:28:43.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:28:43.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:28:43.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:43.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:43.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:43.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_6 2026-02-19T20:28:43.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_6'] 2026-02-19T20:28:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:28:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:43.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:43.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_6 2026-02-19T20:28:43.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:28:43.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:43.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:28:43.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:43.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:43.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:28:43.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:28:43.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:28:43.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:43.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:43.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:43.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_6 2026-02-19T20:28:43.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_6'] 2026-02-19T20:28:43.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:28:43.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:43.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_6 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:28:43.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:43.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:43.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:43.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:43.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:43.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:43.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:43.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:28:43.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:28:43.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:28:43.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:43.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:43.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:43.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_6 2026-02-19T20:28:43.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:28:43.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:28:43.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:28:43.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:28:43.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:28:43.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:28:43.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:28:43.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:43.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_6 2026-02-19T20:28:43.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:43.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_6 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:43.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_6 2026-02-19T20:28:44.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 96 2026-02-19T20:28:44.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_6 2026-02-19T20:28:44.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:44.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:44.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:44.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:44.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:44.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_6.status 2026-02-19T20:28:44.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_6.status 2026-02-19T20:28:44.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:44.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_6.status 2026-02-19T20:28:44.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_6.status 2026-02-19T20:28:44.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_6.status 2026-02-19T20:28:44.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:44.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.status 2026-02-19T20:28:44.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_6 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_6.status 2026-02-19T20:28:44.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:44.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:44.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:44.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_6.status 2026-02-19T20:28:44.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_6 --format xml 2026-02-19T20:28:44.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:44.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:28:44.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=96 2026-02-19T20:28:44.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:44.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:44.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:28:44.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_6 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.status 2026-02-19T20:28:44.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:44.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:44.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:44.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.status 2026-02-19T20:28:44.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_6 --format xml 2026-02-19T20:28:44.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:44.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:44.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=40 2026-02-19T20:28:44.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 96 = 40 2026-02-19T20:28:44.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:44.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:28:45.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_6 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.status 2026-02-19T20:28:45.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:45.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:45.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:45.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.status 2026-02-19T20:28:45.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_6 --format xml 2026-02-19T20:28:45.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:45.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:45.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=40 2026-02-19T20:28:45.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 96 = 40 2026-02-19T20:28:45.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:45.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:28:46.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_6 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.status 2026-02-19T20:28:46.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:46.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:46.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:46.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.status 2026-02-19T20:28:46.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_6 --format xml 2026-02-19T20:28:46.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:46.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:46.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:46.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:28:46.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:28:46.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:28:46.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:28:46.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 96 = '' 2026-02-19T20:28:46.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:46.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:28:47.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_6 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.status 2026-02-19T20:28:47.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:47.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:47.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:47.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.status 2026-02-19T20:28:47.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_6 --format xml 2026-02-19T20:28:47.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:47.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:47.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=96 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 96 = 96 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_6 snap 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:28:47.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:48.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_6@snap 2026-02-19T20:28:48.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_6': 2026-02-19T20:28:48.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:28:48.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:28:48.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:28:48.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113a6be7233 2026-02-19T20:28:48.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113a6be7233 2026-02-19T20:28:48.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:28:48.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:10 2026 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:10 2026 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:10 2026 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 3465736d-6dc5-4652-9dad-a7ea19b6ea48 2026-02-19T20:28:48.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_6 snap 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_6.export 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.export 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_6.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.export 2026-02-19T20:28:48.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_6@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_6.export 2026-02-19T20:28:48.931 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:48.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_6@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.export 2026-02-19T20:28:49.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:49.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_6.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.export 2026-02-19T20:28:49.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_6.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_6.export 2026-02-19T20:28:49.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:49.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:28:49.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_7 2026-02-19T20:28:49.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_7 snap 2026-02-19T20:28:49.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:49.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:49.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:49.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:49.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_7@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:28:49.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_7 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_7 started 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:49.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:49.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:50.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_7 started 2026-02-19T20:28:50.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:50.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:50.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_7 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:28:50.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:50.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:28:50.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:28:50.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:28:50.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_7 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:28:50.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:50.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:50.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:50.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:50.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:50.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:50.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:50.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:28:50.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:28:50.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:28:50.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:50.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:50.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:50.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_7 2026-02-19T20:28:50.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_7'] 2026-02-19T20:28:50.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:28:50.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:50.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:50.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_7 2026-02-19T20:28:50.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:28:50.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:50.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:50.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:28:50.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:28:50.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:28:50.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:28:50.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:28:50.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:50.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:50.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:50.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_7 2026-02-19T20:28:50.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_7'] 2026-02-19T20:28:50.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:28:50.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:50.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:50.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_7 2026-02-19T20:28:50.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:28:50.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:50.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:50.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:28:50.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:28:50.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:50.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:28:50.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:28:50.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:50.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:50.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:50.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_7 2026-02-19T20:28:50.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:28:50.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:28:50.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:28:50.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:28:50.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:28:50.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_7 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_7 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:50.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_7 2026-02-19T20:28:51.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 98 2026-02-19T20:28:51.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_7 2026-02-19T20:28:51.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:51.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:51.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:51.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:51.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:51.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_7.status 2026-02-19T20:28:51.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_7.status 2026-02-19T20:28:51.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:51.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_7.status 2026-02-19T20:28:51.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_7.status 2026-02-19T20:28:51.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_7.status 2026-02-19T20:28:51.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:51.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.status 2026-02-19T20:28:51.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_7 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_7.status 2026-02-19T20:28:51.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:51.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:51.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:51.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_7.status 2026-02-19T20:28:51.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_7 --format xml 2026-02-19T20:28:51.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:51.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:28:51.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=98 2026-02-19T20:28:51.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:51.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:51.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:28:51.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_7 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.status 2026-02-19T20:28:51.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:51.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:51.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:51.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.status 2026-02-19T20:28:51.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_7 --format xml 2026-02-19T20:28:51.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:51.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:51.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=41 2026-02-19T20:28:51.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 98 = 41 2026-02-19T20:28:51.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:51.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:28:52.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_7 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.status 2026-02-19T20:28:52.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:52.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:52.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:52.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.status 2026-02-19T20:28:52.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_7 --format xml 2026-02-19T20:28:52.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:52.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:52.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=41 2026-02-19T20:28:52.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 98 = 41 2026-02-19T20:28:52.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:52.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:28:53.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_7 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.status 2026-02-19T20:28:53.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:53.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:53.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:53.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.status 2026-02-19T20:28:53.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_7 --format xml 2026-02-19T20:28:53.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:53.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:53.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:53.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:28:53.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:28:53.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:28:53.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:28:53.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 98 = '' 2026-02-19T20:28:53.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:53.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:28:53.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2513425978985325) > weight (0.25) 2026-02-19T20:28:53.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.586144370702622) > weight (0.25) 2026-02-19T20:28:53.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2763646068793366) > weight (0.25) 2026-02-19T20:28:53.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6734051872013096) > weight (0.25) 2026-02-19T20:28:53.161 INFO:tasks.rbd_mirror_thrash:waiting for 103.67955843435959 secs before thrashing 2026-02-19T20:28:54.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_7 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.status 2026-02-19T20:28:54.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:54.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:54.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:54.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.status 2026-02-19T20:28:54.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_7 --format xml 2026-02-19T20:28:54.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:54.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:54.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=98 2026-02-19T20:28:54.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 98 = 98 2026-02-19T20:28:54.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:54.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_7 snap 2026-02-19T20:28:54.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:54.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:54.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:54.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:54.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:54.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:28:54.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:55.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_7@snap 2026-02-19T20:28:55.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_7': 2026-02-19T20:28:55.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:28:55.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:28:55.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:28:55.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11dfa846150f 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11dfa846150f 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:16 2026 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:16 2026 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:16 2026 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:28:55.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:28:55.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 5770e33c-f75b-4af2-9f16-1bf9dbb1a48f 2026-02-19T20:28:55.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:28:55.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:55.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_7 snap 2026-02-19T20:28:55.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:55.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:28:55.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:55.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:28:55.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_7.export 2026-02-19T20:28:55.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.export 2026-02-19T20:28:55.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_7.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.export 2026-02-19T20:28:55.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_7@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_7.export 2026-02-19T20:28:55.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:55.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_7@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.export 2026-02-19T20:28:56.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:28:56.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_7.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.export 2026-02-19T20:28:56.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_7.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_7.export 2026-02-19T20:28:56.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:56.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:28:56.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_8 2026-02-19T20:28:56.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_8 snap 2026-02-19T20:28:56.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:56.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:56.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:56.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:28:56.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_8@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:28:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_8 2026-02-19T20:28:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_8 started 2026-02-19T20:28:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:56.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:56.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:56.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:28:56.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:28:56.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:56.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:28:57.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_8 started 2026-02-19T20:28:57.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:57.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:57.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:57.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:28:57.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:28:57.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:28:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_8 2026-02-19T20:28:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:28:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:28:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:28:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:28:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:28:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:28:57.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:28:57.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:28:57.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:28:57.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:57.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_8 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:57.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:57.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:57.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:28:57.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:57.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:28:57.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:28:57.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:57.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:57.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:28:57.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:28:57.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:57.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:57.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:28:57.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_8 2026-02-19T20:28:57.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2026-02-19T20:28:57.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:28:57.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:57.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:57.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_8 2026-02-19T20:28:57.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:28:57.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:28:57.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:57.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:57.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:28:57.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:28:57.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:28:57.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:57.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:57.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:28:57.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_8 2026-02-19T20:28:57.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_8'] 2026-02-19T20:28:57.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:28:57.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:28:57.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:28:57.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_8 2026-02-19T20:28:57.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:28:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:28:57.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:28:57.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:28:57.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:28:57.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:28:57.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:28:57.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:57.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:57.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:28:57.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_8 2026-02-19T20:28:57.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:28:57.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:28:57.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:28:57.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:28:57.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:28:57.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:28:57.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:28:57.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:28:57.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_8 2026-02-19T20:28:57.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:57.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:57.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:28:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:28:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_8 2026-02-19T20:28:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:57.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_8 2026-02-19T20:28:58.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 100 2026-02-19T20:28:58.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_8 2026-02-19T20:28:58.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:28:58.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:58.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:28:58.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:28:58.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:58.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_8.status 2026-02-19T20:28:58.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_8.status 2026-02-19T20:28:58.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:58.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_8.status 2026-02-19T20:28:58.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_8.status 2026-02-19T20:28:58.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_8.status 2026-02-19T20:28:58.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:28:58.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.status 2026-02-19T20:28:58.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_8 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_8.status 2026-02-19T20:28:58.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:28:58.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:58.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:58.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_8.status 2026-02-19T20:28:58.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_8 --format xml 2026-02-19T20:28:58.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:58.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:28:58.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=100 2026-02-19T20:28:58.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:28:58.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:58.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:28:58.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_8 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.status 2026-02-19T20:28:58.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:58.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:58.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:58.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.status 2026-02-19T20:28:58.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_8 --format xml 2026-02-19T20:28:58.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:58.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:58.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=42 2026-02-19T20:28:58.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 100 = 42 2026-02-19T20:28:58.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:58.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:28:59.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_8 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.status 2026-02-19T20:28:59.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:28:59.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:28:59.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:28:59.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.status 2026-02-19T20:28:59.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_8 --format xml 2026-02-19T20:28:59.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:28:59.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:28:59.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=42 2026-02-19T20:28:59.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 100 = 42 2026-02-19T20:28:59.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:28:59.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:00.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_8 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.status 2026-02-19T20:29:00.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:00.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:00.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:29:00.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.status 2026-02-19T20:29:00.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_8 --format xml 2026-02-19T20:29:00.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:00.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:00.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:00.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:00.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:00.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:00.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:00.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 100 = '' 2026-02-19T20:29:00.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:00.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:29:01.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_8 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.status 2026-02-19T20:29:01.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:01.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:01.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:29:01.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.status 2026-02-19T20:29:01.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_8 --format xml 2026-02-19T20:29:01.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:01.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:01.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=100 2026-02-19T20:29:01.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 100 = 100 2026-02-19T20:29:01.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:01.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_8 snap 2026-02-19T20:29:01.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:01.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:01.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:29:01.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:01.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:01.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:29:01.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:02.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_8@snap 2026-02-19T20:29:02.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_8': 2026-02-19T20:29:02.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:29:02.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:29:02.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:29:02.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123d8c19d859 2026-02-19T20:29:02.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123d8c19d859 2026-02-19T20:29:02.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:29:02.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:28 2026 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:28 2026 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:28 2026 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: c9480ce4-7cc9-491d-9e71-9527ddc66985 2026-02-19T20:29:02.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:29:02.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:02.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_8 snap 2026-02-19T20:29:02.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:02.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:29:02.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:02.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:29:02.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_8.export 2026-02-19T20:29:02.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.export 2026-02-19T20:29:02.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_8.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.export 2026-02-19T20:29:02.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_8@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_8.export 2026-02-19T20:29:02.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:02.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_8@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.export 2026-02-19T20:29:03.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:03.080 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_8.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.export 2026-02-19T20:29:03.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_8.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_8.export 2026-02-19T20:29:03.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:03.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:29:03.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_9 2026-02-19T20:29:03.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_9 snap 2026-02-19T20:29:03.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:03.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:03.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:03.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:03.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_9@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:29:03.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_9 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_9 started 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:03.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_9 started 2026-02-19T20:29:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:04.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_9 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:29:04.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:29:04.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:29:04.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:04.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:29:04.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:29:04.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:29:04.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:04.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_9 2026-02-19T20:29:04.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:29:04.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:04.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:04.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:04.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:04.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:04.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:04.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:29:04.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:29:04.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:29:04.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:04.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:04.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:04.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_9 2026-02-19T20:29:04.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2026-02-19T20:29:04.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:04.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:04.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:04.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_9 2026-02-19T20:29:04.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:29:04.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:04.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:29:04.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:04.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:04.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:04.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:04.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:29:04.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:29:04.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:29:04.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:04.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:04.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:04.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_9 2026-02-19T20:29:04.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2026-02-19T20:29:04.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:04.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:04.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:04.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_9 2026-02-19T20:29:04.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:29:04.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:04.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:04.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:29:04.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:29:04.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:04.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:04.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:29:04.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:29:04.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:04.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:04.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:04.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_9 2026-02-19T20:29:04.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_9'] 2026-02-19T20:29:04.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:04.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:04.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:04.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_9 2026-02-19T20:29:04.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:29:04.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:04.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:29:04.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:04.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:29:04.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:29:04.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:29:04.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:29:04.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:29:04.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:29:04.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_9 2026-02-19T20:29:04.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:29:04.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:29:04.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:29:04.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:29:04.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:29:04.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:29:04.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:29:04.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:04.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_9 2026-02-19T20:29:04.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:04.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:04.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:04.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:04.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:04.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:29:04.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:29:04.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_9 2026-02-19T20:29:04.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:04.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:04.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:04.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_9 2026-02-19T20:29:05.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 102 2026-02-19T20:29:05.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_9 2026-02-19T20:29:05.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:05.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:05.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:05.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:05.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:05.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_9.status 2026-02-19T20:29:05.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_9.status 2026-02-19T20:29:05.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:05.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_9.status 2026-02-19T20:29:05.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_9.status 2026-02-19T20:29:05.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_9.status 2026-02-19T20:29:05.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.status 2026-02-19T20:29:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_9 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_9.status 2026-02-19T20:29:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_9.status 2026-02-19T20:29:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_9 --format xml 2026-02-19T20:29:05.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:05.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:29:05.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=102 2026-02-19T20:29:05.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:05.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:05.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:29:05.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_9 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.status 2026-02-19T20:29:05.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:05.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:05.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:05.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.status 2026-02-19T20:29:05.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_9 --format xml 2026-02-19T20:29:05.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:05.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:05.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=43 2026-02-19T20:29:05.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 102 = 43 2026-02-19T20:29:05.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:05.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:29:06.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_9 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.status 2026-02-19T20:29:06.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:06.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:06.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:06.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.status 2026-02-19T20:29:06.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_9 --format xml 2026-02-19T20:29:06.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:06.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:06.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=43 2026-02-19T20:29:06.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 102 = 43 2026-02-19T20:29:06.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:06.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:07.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_9 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.status 2026-02-19T20:29:07.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:07.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:07.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:07.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.status 2026-02-19T20:29:07.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_9 --format xml 2026-02-19T20:29:07.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:07.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:07.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:07.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:07.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:07.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:07.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:07.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 102 = '' 2026-02-19T20:29:07.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:07.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:29:08.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_9 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.status 2026-02-19T20:29:08.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:08.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:08.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:08.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.status 2026-02-19T20:29:08.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_9 --format xml 2026-02-19T20:29:08.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:08.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:08.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=102 2026-02-19T20:29:08.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 102 = 102 2026-02-19T20:29:08.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:08.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_9 snap 2026-02-19T20:29:08.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:08.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:08.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:08.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:08.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:08.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:29:08.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:09.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_9@snap 2026-02-19T20:29:09.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_9': 2026-02-19T20:29:09.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:29:09.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:29:09.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:29:09.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b896f24c8 2026-02-19T20:29:09.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b896f24c8 2026-02-19T20:29:09.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:29:09.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:29:09.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:29:09.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:29:09.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:43 2026 2026-02-19T20:29:09.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:43 2026 2026-02-19T20:29:09.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:43 2026 2026-02-19T20:29:09.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:29:09.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:29:09.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:29:09.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 96e815a6-741f-4db1-bf35-a75f0477bd8e 2026-02-19T20:29:09.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:29:09.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:09.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_9 snap 2026-02-19T20:29:09.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:09.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:29:09.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:09.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:29:09.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_9.export 2026-02-19T20:29:09.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.export 2026-02-19T20:29:09.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_9.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.export 2026-02-19T20:29:09.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_9@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_9.export 2026-02-19T20:29:09.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:09.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_9@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.export 2026-02-19T20:29:10.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:10.083 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_9.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.export 2026-02-19T20:29:10.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_9.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_9.export 2026-02-19T20:29:10.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:10.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:29:10.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_10 2026-02-19T20:29:10.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_10 snap 2026-02-19T20:29:10.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:10.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:10.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:10.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:10.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_10@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:29:10.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_10 2026-02-19T20:29:10.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_10 started 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:10.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:11.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_10 started 2026-02-19T20:29:11.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:11.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:11.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:11.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:29:11.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_10 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:11.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:29:11.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_10 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:11.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:29:11.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:29:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:29:11.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:29:11.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:11.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:11.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:11.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_10 2026-02-19T20:29:11.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2026-02-19T20:29:11.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:11.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:11.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:11.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_10 2026-02-19T20:29:11.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:29:11.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:11.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:11.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:29:11.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:29:11.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:11.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:29:11.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:29:11.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:11.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:11.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:11.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_10 2026-02-19T20:29:11.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_10'] 2026-02-19T20:29:11.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:11.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_10 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:29:11.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:29:11.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:29:11.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:11.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:11.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:11.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_10 2026-02-19T20:29:11.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:29:11.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:29:11.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:29:11.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:29:11.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:29:11.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:29:11.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:29:11.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_10 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_10 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:11.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_10 2026-02-19T20:29:12.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 104 2026-02-19T20:29:12.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_10 2026-02-19T20:29:12.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:12.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:12.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:12.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:12.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:12.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_10.status 2026-02-19T20:29:12.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_10.status 2026-02-19T20:29:12.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:12.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_10.status 2026-02-19T20:29:12.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_10.status 2026-02-19T20:29:12.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_10.status 2026-02-19T20:29:12.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:12.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.status 2026-02-19T20:29:12.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_10 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_10.status 2026-02-19T20:29:12.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:12.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:12.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:12.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_10.status 2026-02-19T20:29:12.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_10 --format xml 2026-02-19T20:29:12.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:12.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:29:12.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=104 2026-02-19T20:29:12.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:12.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:12.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:29:12.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_10 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.status 2026-02-19T20:29:12.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:12.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:12.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:12.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.status 2026-02-19T20:29:12.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_10 --format xml 2026-02-19T20:29:12.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:12.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:12.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=44 2026-02-19T20:29:12.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 104 = 44 2026-02-19T20:29:12.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:12.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:29:13.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_10 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.status 2026-02-19T20:29:13.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:13.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:13.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:13.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.status 2026-02-19T20:29:13.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_10 --format xml 2026-02-19T20:29:13.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:13.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:13.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=44 2026-02-19T20:29:13.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 104 = 44 2026-02-19T20:29:13.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:13.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:14.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_10 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.status 2026-02-19T20:29:14.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:14.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:14.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:14.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.status 2026-02-19T20:29:14.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_10 --format xml 2026-02-19T20:29:14.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:14.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:14.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:14.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:14.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:14.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:14.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:14.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 104 = '' 2026-02-19T20:29:14.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:14.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:29:15.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_10 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.status 2026-02-19T20:29:15.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:15.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:15.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:15.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.status 2026-02-19T20:29:15.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_10 --format xml 2026-02-19T20:29:15.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:15.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=104 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 104 = 104 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_10 snap 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:29:15.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:16.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_10@snap 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_10': 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113a2a97f142 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113a2a97f142 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:19:57 2026 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:19:57 2026 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:19:57 2026 2026-02-19T20:29:16.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:29:16.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:29:16.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:29:16.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: b4b55785-9125-4cba-be08-1b555be092a3 2026-02-19T20:29:16.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_10 snap 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_10.export 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.export 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_10.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.export 2026-02-19T20:29:16.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_10@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_10.export 2026-02-19T20:29:16.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:16.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_10@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.export 2026-02-19T20:29:17.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:17.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_10.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.export 2026-02-19T20:29:17.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_10.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_10.export 2026-02-19T20:29:17.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:17.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:29:17.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_11 2026-02-19T20:29:17.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_11 snap 2026-02-19T20:29:17.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:17.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:17.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:17.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:17.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_11@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:29:17.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_11 2026-02-19T20:29:17.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:17.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:17.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:17.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_11 started 2026-02-19T20:29:17.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:17.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:29:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:17.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:18.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_11 started 2026-02-19T20:29:18.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:29:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:29:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:29:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_11 2026-02-19T20:29:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:29:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:18.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:29:18.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:29:18.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:29:18.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:29:18.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:18.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:29:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:29:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:29:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_11 2026-02-19T20:29:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:29:18.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:29:18.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:18.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:18.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:18.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:29:18.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:29:18.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:29:18.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:18.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:18.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:18.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_11 2026-02-19T20:29:18.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_11'] 2026-02-19T20:29:18.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:18.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:18.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:18.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_11 2026-02-19T20:29:18.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:18.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:18.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:18.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:18.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:18.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:18.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:29:18.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:29:18.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:29:18.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:18.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:18.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:18.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_11 2026-02-19T20:29:18.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_11'] 2026-02-19T20:29:18.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:18.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:18.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:18.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_11 2026-02-19T20:29:18.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:29:18.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:18.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:18.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:29:18.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:29:18.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:29:18.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:29:18.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:18.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:18.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:18.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_11 2026-02-19T20:29:18.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:29:18.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:29:18.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:29:18.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:29:18.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:29:18.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_11 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_11 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:18.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_11 2026-02-19T20:29:19.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 106 2026-02-19T20:29:19.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_11 2026-02-19T20:29:19.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:19.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:19.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:19.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:19.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:19.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_11.status 2026-02-19T20:29:19.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_11.status 2026-02-19T20:29:19.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:19.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_11.status 2026-02-19T20:29:19.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_11.status 2026-02-19T20:29:19.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_11.status 2026-02-19T20:29:19.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:19.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.status 2026-02-19T20:29:19.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_11 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_11.status 2026-02-19T20:29:19.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:19.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:19.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:19.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_11.status 2026-02-19T20:29:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_11 --format xml 2026-02-19T20:29:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:19.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:29:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=106 2026-02-19T20:29:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:19.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:29:19.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_11 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.status 2026-02-19T20:29:19.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:19.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:19.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:19.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.status 2026-02-19T20:29:19.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_11 --format xml 2026-02-19T20:29:19.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:19.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:19.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=45 2026-02-19T20:29:19.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 106 = 45 2026-02-19T20:29:19.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:19.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:29:20.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_11 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.status 2026-02-19T20:29:20.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:20.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:20.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:20.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.status 2026-02-19T20:29:20.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_11 --format xml 2026-02-19T20:29:20.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:20.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=45 2026-02-19T20:29:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 106 = 45 2026-02-19T20:29:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:21.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_11 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.status 2026-02-19T20:29:21.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:21.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:21.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:21.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.status 2026-02-19T20:29:21.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_11 --format xml 2026-02-19T20:29:21.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:21.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:21.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:21.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:21.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:21.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:21.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:21.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 106 = '' 2026-02-19T20:29:21.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:21.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:29:22.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_11 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.status 2026-02-19T20:29:22.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:22.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:22.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:22.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.status 2026-02-19T20:29:22.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_11 --format xml 2026-02-19T20:29:22.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:22.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:22.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=106 2026-02-19T20:29:22.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 106 = 106 2026-02-19T20:29:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_11 snap 2026-02-19T20:29:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:29:22.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:23.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_11@snap 2026-02-19T20:29:23.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_11': 2026-02-19T20:29:23.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:29:23.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11df3a52fc62 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11df3a52fc62 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:05 2026 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:05 2026 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:05 2026 2026-02-19T20:29:23.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:29:23.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:29:23.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:29:23.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: d20ca347-64b1-43f6-9ef4-b6887cc47d4b 2026-02-19T20:29:23.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:29:23.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:23.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_11 snap 2026-02-19T20:29:23.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:23.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:29:23.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:23.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:29:23.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_11.export 2026-02-19T20:29:23.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.export 2026-02-19T20:29:23.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_11.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.export 2026-02-19T20:29:23.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_11@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_11.export 2026-02-19T20:29:23.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:23.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_11@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.export 2026-02-19T20:29:24.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:24.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_11.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.export 2026-02-19T20:29:24.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_11.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_11.export 2026-02-19T20:29:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:29:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_12 2026-02-19T20:29:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_12 snap 2026-02-19T20:29:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_12@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:29:24.490 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5228075563949077) > weight (0.25) 2026-02-19T20:29:24.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7805330013877025) > weight (0.25) 2026-02-19T20:29:24.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7035170970369122) > weight (0.25) 2026-02-19T20:29:24.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5247476786518144) > weight (0.25) 2026-02-19T20:29:24.491 INFO:tasks.rbd_mirror_thrash:waiting for 78.23281120535748 secs before thrashing 2026-02-19T20:29:24.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_12 2026-02-19T20:29:24.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:24.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_12 started 2026-02-19T20:29:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:29:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:24.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_12 started 2026-02-19T20:29:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:25.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_12 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:29:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:29:25.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:29:25.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_12 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:25.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:29:25.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:29:25.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:25.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:25.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:25.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_12 2026-02-19T20:29:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_12'] 2026-02-19T20:29:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:25.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_12 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:25.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:29:25.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:29:25.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:29:25.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:25.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:25.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:25.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_12 2026-02-19T20:29:25.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_12'] 2026-02-19T20:29:25.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:25.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:25.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:25.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_12 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:25.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:25.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:25.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:25.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:25.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:25.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:25.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:25.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:29:25.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:29:25.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:29:25.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:25.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:25.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:25.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_12 2026-02-19T20:29:25.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:29:25.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:29:25.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:29:25.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:29:25.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:29:25.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:29:25.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_12 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_12 2026-02-19T20:29:25.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:25.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:25.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:25.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_12 2026-02-19T20:29:26.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 108 2026-02-19T20:29:26.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_12 2026-02-19T20:29:26.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:26.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:26.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:26.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:26.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:26.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_12.status 2026-02-19T20:29:26.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_12.status 2026-02-19T20:29:26.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:26.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_12.status 2026-02-19T20:29:26.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_12.status 2026-02-19T20:29:26.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_12.status 2026-02-19T20:29:26.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:26.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.status 2026-02-19T20:29:26.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_12 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_12.status 2026-02-19T20:29:26.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:26.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:26.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:26.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_12.status 2026-02-19T20:29:26.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_12 --format xml 2026-02-19T20:29:26.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:26.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:29:26.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=108 2026-02-19T20:29:26.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:26.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:26.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:29:26.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_12 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.status 2026-02-19T20:29:26.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:26.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:26.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:26.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.status 2026-02-19T20:29:26.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_12 --format xml 2026-02-19T20:29:26.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:26.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:26.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=46 2026-02-19T20:29:26.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 108 = 46 2026-02-19T20:29:26.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:26.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:29:27.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_12 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.status 2026-02-19T20:29:27.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:27.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:27.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:27.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.status 2026-02-19T20:29:27.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_12 --format xml 2026-02-19T20:29:27.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:27.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:27.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=46 2026-02-19T20:29:27.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 108 = 46 2026-02-19T20:29:27.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:27.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:28.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_12 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.status 2026-02-19T20:29:28.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:28.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:28.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:28.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.status 2026-02-19T20:29:28.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_12 --format xml 2026-02-19T20:29:28.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:28.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:28.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:28.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:28.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:28.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:28.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:28.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 108 = '' 2026-02-19T20:29:28.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:28.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:29:29.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_12 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.status 2026-02-19T20:29:29.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:29.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:29.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:29.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.status 2026-02-19T20:29:29.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_12 --format xml 2026-02-19T20:29:29.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:29.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:29.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=108 2026-02-19T20:29:29.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 108 = 108 2026-02-19T20:29:29.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:29.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_12 snap 2026-02-19T20:29:29.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:29.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:29.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:29.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:29.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:29.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:29:29.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:30.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_12@snap 2026-02-19T20:29:30.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_12': 2026-02-19T20:29:30.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:29:30.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:29:30.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:29:30.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123d2088a2cd 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123d2088a2cd 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:20 2026 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:20 2026 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:20 2026 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:29:30.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:29:30.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:29:30.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: a442b5dd-49c6-40c0-a740-73e9245e620d 2026-02-19T20:29:30.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:29:30.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:30.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_12 snap 2026-02-19T20:29:30.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:30.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:29:30.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:30.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:29:30.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_12.export 2026-02-19T20:29:30.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.export 2026-02-19T20:29:30.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_12.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.export 2026-02-19T20:29:30.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_12@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_12.export 2026-02-19T20:29:30.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:30.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_12@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.export 2026-02-19T20:29:31.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:31.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_12.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.export 2026-02-19T20:29:31.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_12.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_12.export 2026-02-19T20:29:31.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:31.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:29:31.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_13 2026-02-19T20:29:31.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_13 snap 2026-02-19T20:29:31.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:31.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:31.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:31.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:31.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_13@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:29:31.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_13 2026-02-19T20:29:31.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_13 started 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:31.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:32.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_13 started 2026-02-19T20:29:32.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_13 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:32.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:29:32.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:29:32.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_13 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:29:32.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:29:32.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:29:32.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:32.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:32.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:32.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_13 2026-02-19T20:29:32.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_13'] 2026-02-19T20:29:32.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:32.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_13 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:32.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:29:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:29:32.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:32.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:32.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:32.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_13 2026-02-19T20:29:32.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:29:32.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:29:32.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:29:32.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:29:32.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:29:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:29:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:29:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_13 2026-02-19T20:29:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:32.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:32.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:29:32.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:29:32.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_13 2026-02-19T20:29:32.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:32.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:32.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:32.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_13 2026-02-19T20:29:33.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 110 2026-02-19T20:29:33.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_13 2026-02-19T20:29:33.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:33.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:33.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:33.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:33.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:33.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_13.status 2026-02-19T20:29:33.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_13.status 2026-02-19T20:29:33.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:33.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_13.status 2026-02-19T20:29:33.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_13.status 2026-02-19T20:29:33.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_13.status 2026-02-19T20:29:33.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:33.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.status 2026-02-19T20:29:33.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_13 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_13.status 2026-02-19T20:29:33.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:33.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:33.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:33.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_13.status 2026-02-19T20:29:33.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_13 --format xml 2026-02-19T20:29:33.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:33.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:29:33.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=110 2026-02-19T20:29:33.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:33.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:33.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:29:33.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_13 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.status 2026-02-19T20:29:33.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:33.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:33.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:33.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.status 2026-02-19T20:29:33.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_13 --format xml 2026-02-19T20:29:33.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:33.932 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:33.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=47 2026-02-19T20:29:33.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 110 = 47 2026-02-19T20:29:33.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:33.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:29:34.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_13 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.status 2026-02-19T20:29:34.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:34.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:34.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:34.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.status 2026-02-19T20:29:34.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_13 --format xml 2026-02-19T20:29:34.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:34.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:34.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=47 2026-02-19T20:29:34.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 110 = 47 2026-02-19T20:29:34.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:34.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:35.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_13 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.status 2026-02-19T20:29:35.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:35.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:35.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:35.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.status 2026-02-19T20:29:35.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_13 --format xml 2026-02-19T20:29:35.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:35.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:35.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:35.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:35.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:35.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:35.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:35.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 110 = '' 2026-02-19T20:29:35.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:35.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:29:36.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_13 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.status 2026-02-19T20:29:36.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:36.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:36.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:36.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.status 2026-02-19T20:29:36.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_13 --format xml 2026-02-19T20:29:36.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:36.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:36.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=110 2026-02-19T20:29:36.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 110 = 110 2026-02-19T20:29:36.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:36.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_13 snap 2026-02-19T20:29:36.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:36.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:36.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:36.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:36.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:36.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:29:36.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:37.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_13@snap 2026-02-19T20:29:37.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_13': 2026-02-19T20:29:37.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:29:37.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:29:37.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b8df128b87 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b8df128b87 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:29 2026 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:29 2026 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:29 2026 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:29:37.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:29:37.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: b748d40b-d69e-4188-bd35-09d03d9a991d 2026-02-19T20:29:37.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:29:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_13 snap 2026-02-19T20:29:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:29:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:29:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_13.export 2026-02-19T20:29:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.export 2026-02-19T20:29:37.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_13.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.export 2026-02-19T20:29:37.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_13@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_13.export 2026-02-19T20:29:37.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:37.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_13@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.export 2026-02-19T20:29:38.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:38.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_13.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.export 2026-02-19T20:29:38.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_13.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_13.export 2026-02-19T20:29:38.230 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:38.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:29:38.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_14 2026-02-19T20:29:38.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_14 snap 2026-02-19T20:29:38.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:38.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:38.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:38.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:38.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_14@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_14 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_14 started 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:38.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:39.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_14 started 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_14 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:39.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:29:39.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:29:39.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:29:39.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:29:39.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:39.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:29:39.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:29:39.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:29:39.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_14 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:29:39.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:29:39.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:39.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:39.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:39.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:39.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:39.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:39.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:39.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:29:39.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:29:39.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:29:39.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:39.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:39.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:39.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_14 2026-02-19T20:29:39.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2026-02-19T20:29:39.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:39.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:39.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:39.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_14 2026-02-19T20:29:39.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:29:39.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:39.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:29:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:39.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:39.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:39.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:29:39.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:29:39.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:29:39.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:39.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:39.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:39.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_14 2026-02-19T20:29:39.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2026-02-19T20:29:39.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:39.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:39.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:39.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_14 2026-02-19T20:29:39.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:39.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:29:39.435 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:29:39.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:39.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:39.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:29:39.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:29:39.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:29:39.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:39.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:39.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:29:39.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_14 2026-02-19T20:29:39.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_14'] 2026-02-19T20:29:39.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:39.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_14 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:29:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:29:39.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:29:39.494 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:29:39.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:29:39.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:29:39.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_14 2026-02-19T20:29:39.542 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:29:39.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:29:39.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:29:39.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:29:39.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:29:39.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:29:39.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:29:39.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:39.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_14 2026-02-19T20:29:39.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:39.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_14 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:39.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_14 2026-02-19T20:29:40.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 112 2026-02-19T20:29:40.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_14 2026-02-19T20:29:40.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:40.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:40.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:40.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:40.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:40.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_14.status 2026-02-19T20:29:40.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_14.status 2026-02-19T20:29:40.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:40.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_14.status 2026-02-19T20:29:40.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_14.status 2026-02-19T20:29:40.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_14.status 2026-02-19T20:29:40.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:40.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.status 2026-02-19T20:29:40.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_14 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_14.status 2026-02-19T20:29:40.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:40.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:40.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:40.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_14.status 2026-02-19T20:29:40.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_14 --format xml 2026-02-19T20:29:40.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:40.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:29:40.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=112 2026-02-19T20:29:40.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:40.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:40.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:29:40.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_14 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.status 2026-02-19T20:29:40.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:40.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:40.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:40.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.status 2026-02-19T20:29:40.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_14 --format xml 2026-02-19T20:29:40.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:40.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:40.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:40.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:40.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:40.594 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:40.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:40.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 112 = '' 2026-02-19T20:29:40.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:40.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:29:40.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_14 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.status 2026-02-19T20:29:40.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:40.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:40.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:40.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.status 2026-02-19T20:29:40.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_14 --format xml 2026-02-19T20:29:40.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:41.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:41.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:41.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:41.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:41.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:41.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:41.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 112 = '' 2026-02-19T20:29:41.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:41.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:41.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_14 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.status 2026-02-19T20:29:41.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:41.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:41.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:41.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.status 2026-02-19T20:29:41.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_14 --format xml 2026-02-19T20:29:41.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:41.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:41.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=112 2026-02-19T20:29:41.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 112 = 112 2026-02-19T20:29:41.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:41.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_14 snap 2026-02-19T20:29:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:29:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:42.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_14@snap 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_14': 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113abf797411 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113abf797411 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:29:42.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:44 2026 2026-02-19T20:29:42.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:44 2026 2026-02-19T20:29:42.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:44 2026 2026-02-19T20:29:42.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:29:42.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:29:42.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:29:42.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 47d7aec0-eb62-49a0-9f34-a0c1ec276d42 2026-02-19T20:29:42.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_14 snap 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_14.export 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.export 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_14.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.export 2026-02-19T20:29:42.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_14@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_14.export 2026-02-19T20:29:43.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:43.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_14@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.export 2026-02-19T20:29:43.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:43.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_14.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.export 2026-02-19T20:29:43.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_14.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_14.export 2026-02-19T20:29:43.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:43.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:29:43.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_15 2026-02-19T20:29:43.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_15 snap 2026-02-19T20:29:43.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:43.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:43.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:43.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:43.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_15@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:29:43.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_15 2026-02-19T20:29:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_15 started 2026-02-19T20:29:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:29:43.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:43.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:43.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:44.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_15 started 2026-02-19T20:29:44.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:44.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:44.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:44.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:29:44.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:29:44.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:29:44.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_15 2026-02-19T20:29:44.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:29:44.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:29:44.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:44.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:29:44.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:29:44.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:29:44.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:44.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:29:44.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:29:44.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:29:44.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:44.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_15 2026-02-19T20:29:44.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:29:44.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:44.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:44.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:29:44.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:29:44.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:44.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:44.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:44.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:29:44.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:44.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:44.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:44.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:29:44.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:44.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:29:44.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:29:44.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:44.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:44.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:29:44.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:29:44.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:44.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:44.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:44.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_15 2026-02-19T20:29:44.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_15'] 2026-02-19T20:29:44.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:44.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:44.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:44.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_15 2026-02-19T20:29:44.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:29:44.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:44.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:44.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:29:44.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:29:44.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:29:44.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:29:44.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:29:44.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:44.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:44.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:44.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_15 2026-02-19T20:29:44.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:29:44.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:29:44.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:29:44.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:29:44.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:29:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:29:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:29:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_15 2026-02-19T20:29:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:44.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:44.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:29:44.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:29:44.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_15 2026-02-19T20:29:44.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:44.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:44.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:44.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_15 2026-02-19T20:29:45.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 114 2026-02-19T20:29:45.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_15 2026-02-19T20:29:45.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:45.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:45.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:45.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:45.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:45.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_15.status 2026-02-19T20:29:45.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_15.status 2026-02-19T20:29:45.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:45.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_15.status 2026-02-19T20:29:45.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_15.status 2026-02-19T20:29:45.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_15.status 2026-02-19T20:29:45.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.status 2026-02-19T20:29:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_15 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_15.status 2026-02-19T20:29:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_15.status 2026-02-19T20:29:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_15 --format xml 2026-02-19T20:29:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:45.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:29:45.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=114 2026-02-19T20:29:45.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:45.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:45.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:29:45.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_15 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.status 2026-02-19T20:29:45.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:45.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.status 2026-02-19T20:29:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_15 --format xml 2026-02-19T20:29:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:45.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:45.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=49 2026-02-19T20:29:45.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 114 = 49 2026-02-19T20:29:45.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:45.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:29:46.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_15 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.status 2026-02-19T20:29:46.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:46.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:46.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:46.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.status 2026-02-19T20:29:46.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_15 --format xml 2026-02-19T20:29:46.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:46.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:46.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=49 2026-02-19T20:29:46.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 114 = 49 2026-02-19T20:29:46.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:46.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:47.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_15 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.status 2026-02-19T20:29:47.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:47.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:47.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:47.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.status 2026-02-19T20:29:47.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_15 --format xml 2026-02-19T20:29:47.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:47.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:47.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:47.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:47.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:47.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:47.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:47.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 114 = '' 2026-02-19T20:29:47.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:47.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:29:48.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_15 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.status 2026-02-19T20:29:48.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:48.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:48.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:48.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.status 2026-02-19T20:29:48.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_15 --format xml 2026-02-19T20:29:48.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:48.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:48.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=114 2026-02-19T20:29:48.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 114 = 114 2026-02-19T20:29:48.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:48.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_15 snap 2026-02-19T20:29:48.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:48.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:48.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:48.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:48.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:48.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:29:48.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:49.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_15@snap 2026-02-19T20:29:49.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_15': 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11dfd4fdfdc7 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11dfd4fdfdc7 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:29:49.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:20:53 2026 2026-02-19T20:29:49.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:20:53 2026 2026-02-19T20:29:49.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:20:53 2026 2026-02-19T20:29:49.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:29:49.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:29:49.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:29:49.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: c0ee1cc7-c443-4e99-9bcc-31e4eda9076d 2026-02-19T20:29:49.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:29:49.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:49.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_15 snap 2026-02-19T20:29:49.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:49.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:29:49.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:49.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:29:49.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_15.export 2026-02-19T20:29:49.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.export 2026-02-19T20:29:49.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_15.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.export 2026-02-19T20:29:49.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_15@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_15.export 2026-02-19T20:29:49.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:49.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_15@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.export 2026-02-19T20:29:50.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:50.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_15.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.export 2026-02-19T20:29:50.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_15.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_15.export 2026-02-19T20:29:50.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:50.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:29:50.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_16 2026-02-19T20:29:50.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_16 snap 2026-02-19T20:29:50.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:50.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:50.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:50.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:50.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_16@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:29:50.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_16 2026-02-19T20:29:50.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:50.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:50.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:50.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_16 started 2026-02-19T20:29:50.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:50.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:50.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:50.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:29:50.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:50.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:50.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:51.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_16 started 2026-02-19T20:29:51.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:51.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:51.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:51.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:29:51.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:29:51.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:29:51.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_16 2026-02-19T20:29:51.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:29:51.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:29:51.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:51.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:29:51.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:29:51.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:29:51.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:51.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_16 2026-02-19T20:29:51.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:29:51.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:29:51.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:51.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:51.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:51.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:29:51.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:29:51.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:29:51.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:51.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:51.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:51.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_16 2026-02-19T20:29:51.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:29:51.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:29:51.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:29:51.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:29:51.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:29:51.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:29:51.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:29:51.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:51.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_16 2026-02-19T20:29:51.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:51.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:51.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:51.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:51.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:51.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:29:51.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:29:51.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_16 2026-02-19T20:29:51.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:51.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:51.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:51.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_16 2026-02-19T20:29:52.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 116 2026-02-19T20:29:52.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_16 2026-02-19T20:29:52.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:52.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:52.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:52.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:52.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:52.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_16.status 2026-02-19T20:29:52.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_16.status 2026-02-19T20:29:52.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:52.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_16.status 2026-02-19T20:29:52.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_16.status 2026-02-19T20:29:52.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_16.status 2026-02-19T20:29:52.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:52.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.status 2026-02-19T20:29:52.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_16 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_16.status 2026-02-19T20:29:52.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:52.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:52.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:52.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_16.status 2026-02-19T20:29:52.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_16 --format xml 2026-02-19T20:29:52.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:52.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:29:52.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=116 2026-02-19T20:29:52.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:52.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:52.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:29:52.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_16 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.status 2026-02-19T20:29:52.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:52.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:52.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:52.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.status 2026-02-19T20:29:52.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_16 --format xml 2026-02-19T20:29:52.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:52.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:52.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:52.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:52.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:52.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:52.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:52.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 116 = '' 2026-02-19T20:29:52.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:52.588 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:29:52.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_16 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.status 2026-02-19T20:29:52.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:52.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:52.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:52.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.status 2026-02-19T20:29:52.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_16 --format xml 2026-02-19T20:29:52.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:53.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:53.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:53.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:53.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:53.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:53.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:53.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 116 = '' 2026-02-19T20:29:53.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:53.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:53.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_16 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.status 2026-02-19T20:29:53.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:53.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:53.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:53.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.status 2026-02-19T20:29:53.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_16 --format xml 2026-02-19T20:29:53.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:53.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:53.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=116 2026-02-19T20:29:53.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 116 = 116 2026-02-19T20:29:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_16 snap 2026-02-19T20:29:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:29:53.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:54.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_16@snap 2026-02-19T20:29:54.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_16': 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123dee6a5db 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123dee6a5db 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:06 2026 2026-02-19T20:29:54.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:06 2026 2026-02-19T20:29:54.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:06 2026 2026-02-19T20:29:54.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:29:54.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:29:54.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:29:54.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 97f2f53e-69ec-439c-87a1-d3ef7d099edc 2026-02-19T20:29:54.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:29:54.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:54.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_16 snap 2026-02-19T20:29:54.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:54.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:29:54.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:54.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:29:54.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_16.export 2026-02-19T20:29:54.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.export 2026-02-19T20:29:54.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_16.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.export 2026-02-19T20:29:54.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_16@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_16.export 2026-02-19T20:29:54.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:55.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_16@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.export 2026-02-19T20:29:55.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:29:55.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_16.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.export 2026-02-19T20:29:55.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_16.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_16.export 2026-02-19T20:29:55.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:55.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:29:55.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_17 2026-02-19T20:29:55.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_17 snap 2026-02-19T20:29:55.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:55.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:55.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:55.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:29:55.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_17@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:29:55.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_17 2026-02-19T20:29:55.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:55.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:55.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:55.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_17 started 2026-02-19T20:29:55.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:55.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:55.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:55.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:29:55.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:29:55.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:55.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:29:56.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_17 started 2026-02-19T20:29:56.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:56.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:56.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:56.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_17 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:29:56.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:29:56.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:29:56.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:29:56.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:56.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_17 2026-02-19T20:29:56.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:56.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:56.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:29:56.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:29:56.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:56.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:56.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:29:56.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:29:56.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:29:56.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:56.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:56.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:29:56.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_17 2026-02-19T20:29:56.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_17'] 2026-02-19T20:29:56.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:29:56.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:29:56.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:29:56.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_17 2026-02-19T20:29:56.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:29:56.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:29:56.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:29:56.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:29:56.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:29:56.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:29:56.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:29:56.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:29:56.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:29:56.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:56.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:56.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:29:56.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_17 2026-02-19T20:29:56.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:29:56.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:29:56.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:29:56.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:29:56.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:29:56.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:29:56.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:29:56.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:29:56.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_17 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_17 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:56.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_17 2026-02-19T20:29:57.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 118 2026-02-19T20:29:57.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_17 2026-02-19T20:29:57.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:29:57.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:57.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:29:57.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:29:57.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:57.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_17.status 2026-02-19T20:29:57.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_17.status 2026-02-19T20:29:57.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:57.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_17.status 2026-02-19T20:29:57.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_17.status 2026-02-19T20:29:57.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_17.status 2026-02-19T20:29:57.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:29:57.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.status 2026-02-19T20:29:57.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_17 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_17.status 2026-02-19T20:29:57.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:29:57.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:57.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:57.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_17.status 2026-02-19T20:29:57.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_17 --format xml 2026-02-19T20:29:57.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:57.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:29:57.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=118 2026-02-19T20:29:57.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:57.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:57.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:29:57.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_17 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.status 2026-02-19T20:29:57.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:57.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:57.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:57.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.status 2026-02-19T20:29:57.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_17 --format xml 2026-02-19T20:29:57.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:57.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:57.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=51 2026-02-19T20:29:57.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 118 = 51 2026-02-19T20:29:57.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:57.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:29:58.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_17 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.status 2026-02-19T20:29:58.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:58.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:58.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:58.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.status 2026-02-19T20:29:58.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_17 --format xml 2026-02-19T20:29:58.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:58.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:58.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=51 2026-02-19T20:29:58.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 118 = 51 2026-02-19T20:29:58.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:58.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:29:59.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_17 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.status 2026-02-19T20:29:59.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:29:59.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:29:59.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:29:59.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.status 2026-02-19T20:29:59.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_17 --format xml 2026-02-19T20:29:59.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:29:59.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:29:59.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:29:59.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:29:59.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:29:59.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:29:59.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:29:59.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 118 = '' 2026-02-19T20:29:59.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:29:59.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:30:00.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_17 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.status 2026-02-19T20:30:00.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:00.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:00.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:30:00.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.status 2026-02-19T20:30:00.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_17 --format xml 2026-02-19T20:30:00.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:00.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:00.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=118 2026-02-19T20:30:00.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 118 = 118 2026-02-19T20:30:00.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:00.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_17 snap 2026-02-19T20:30:00.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:00.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:00.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:30:00.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:00.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:00.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:00.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:01.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_17@snap 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_17': 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b817c1fb31 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b817c1fb31 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:20 2026 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:20 2026 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:20 2026 2026-02-19T20:30:01.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:01.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:01.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:01.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 412c65be-fc7c-4de7-ab85-528c3d9910a2 2026-02-19T20:30:01.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_17 snap 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_17.export 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.export 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_17.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.export 2026-02-19T20:30:01.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_17@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_17.export 2026-02-19T20:30:01.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:02.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_17@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.export 2026-02-19T20:30:02.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:02.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_17.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.export 2026-02-19T20:30:02.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_17.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_17.export 2026-02-19T20:30:02.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:02.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:02.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_18 2026-02-19T20:30:02.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_18 snap 2026-02-19T20:30:02.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:02.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:02.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:02.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:02.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_18@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:02.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_18 2026-02-19T20:30:02.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:02.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:02.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:02.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_18 started 2026-02-19T20:30:02.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:02.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:02.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:02.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:02.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:02.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:02.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:03.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_18 started 2026-02-19T20:30:03.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:03.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:03.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:03.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:03.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:03.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_18 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:03.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:03.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:03.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:03.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_18 2026-02-19T20:30:03.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:03.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:03.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:03.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:03.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:03.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:03.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:03.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:03.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:03.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:03.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:03.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_18 2026-02-19T20:30:03.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_18'] 2026-02-19T20:30:03.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:03.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:03.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:03.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_18 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:03.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:03.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:03.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:30:03.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:03.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:30:03.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:30:03.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:03.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:30:03.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:30:03.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:30:03.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:03.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:03.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:03.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_18 2026-02-19T20:30:03.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_18'] 2026-02-19T20:30:03.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:03.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:03.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:03.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_18 2026-02-19T20:30:03.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:30:03.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:03.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:03.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:30:03.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:30:03.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:03.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:03.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:03.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:30:03.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:30:03.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:03.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:03.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:30:03.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:30:03.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:03.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:03.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:30:03.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:30:03.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:30:03.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:30:03.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:30:03.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:30:03.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_18 2026-02-19T20:30:03.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_18'] 2026-02-19T20:30:03.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:03.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:03.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:03.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_18 2026-02-19T20:30:03.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:30:03.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:03.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:03.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:30:03.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:30:03.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:30:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:30:03.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:30:03.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:30:03.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:30:03.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_18 2026-02-19T20:30:03.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:03.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:03.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_18 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:03.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:03.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:03.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_18 2026-02-19T20:30:03.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:03.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:03.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:03.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_18 2026-02-19T20:30:04.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 120 2026-02-19T20:30:04.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_18 2026-02-19T20:30:04.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:04.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:04.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:04.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:04.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:04.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_18.status 2026-02-19T20:30:04.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_18.status 2026-02-19T20:30:04.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:04.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_18.status 2026-02-19T20:30:04.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_18.status 2026-02-19T20:30:04.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_18.status 2026-02-19T20:30:04.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:04.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.status 2026-02-19T20:30:04.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_18 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_18.status 2026-02-19T20:30:04.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:04.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:04.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:04.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_18.status 2026-02-19T20:30:04.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_18 --format xml 2026-02-19T20:30:04.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:04.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:04.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=120 2026-02-19T20:30:04.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:04.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:04.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:04.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_18 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.status 2026-02-19T20:30:04.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:04.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:04.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:04.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.status 2026-02-19T20:30:04.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_18 --format xml 2026-02-19T20:30:04.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:04.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:04.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:04.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:04.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:04.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:04.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:04.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 120 = '' 2026-02-19T20:30:04.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:04.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:30:04.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_18 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.status 2026-02-19T20:30:04.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:04.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:04.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:04.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.status 2026-02-19T20:30:04.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_18 --format xml 2026-02-19T20:30:04.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:05.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:05.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:05.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:05.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:05.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:05.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:05.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 120 = '' 2026-02-19T20:30:05.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:05.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:30:05.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_18 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.status 2026-02-19T20:30:05.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:05.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:05.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:05.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.status 2026-02-19T20:30:05.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_18 --format xml 2026-02-19T20:30:05.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:05.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:05.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=120 2026-02-19T20:30:05.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 120 = 120 2026-02-19T20:30:05.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:05.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_18 snap 2026-02-19T20:30:05.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:05.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:05.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:05.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:05.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:05.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:05.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:06.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_18@snap 2026-02-19T20:30:06.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_18': 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113aa96f0aa3 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113aa96f0aa3 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:06.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:35 2026 2026-02-19T20:30:06.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:35 2026 2026-02-19T20:30:06.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:35 2026 2026-02-19T20:30:06.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:06.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:06.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:06.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: b08bc81f-704a-41ce-b79c-ab730842d310 2026-02-19T20:30:06.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:06.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:06.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_18 snap 2026-02-19T20:30:06.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:06.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:30:06.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:06.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:06.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_18.export 2026-02-19T20:30:06.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.export 2026-02-19T20:30:06.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_18.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.export 2026-02-19T20:30:06.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_18@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_18.export 2026-02-19T20:30:07.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:07.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_18@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.export 2026-02-19T20:30:07.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:07.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_18.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.export 2026-02-19T20:30:07.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_18.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_18.export 2026-02-19T20:30:07.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:07.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:07.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_19 2026-02-19T20:30:07.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_19 snap 2026-02-19T20:30:07.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:07.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:07.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:07.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:07.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_19@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_19 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_19 started 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:07.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_19 started 2026-02-19T20:30:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_19 2026-02-19T20:30:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:08.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:08.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_19 2026-02-19T20:30:08.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:08.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:08.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:08.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:08.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:08.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:08.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:08.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:08.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:08.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:08.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:08.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_19 2026-02-19T20:30:08.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_19'] 2026-02-19T20:30:08.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:08.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_19 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:08.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:08.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:30:08.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:30:08.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:30:08.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:08.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:08.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:08.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_19 2026-02-19T20:30:08.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:08.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:08.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:08.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:08.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:08.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:08.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:08.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:08.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_19 2026-02-19T20:30:08.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:08.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:08.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:08.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:08.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:08.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:08.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:08.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_19 2026-02-19T20:30:08.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:08.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:08.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:08.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_19 2026-02-19T20:30:09.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 122 2026-02-19T20:30:09.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_19 2026-02-19T20:30:09.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:09.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:09.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:09.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:09.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:09.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_19.status 2026-02-19T20:30:09.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_19.status 2026-02-19T20:30:09.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:09.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_19.status 2026-02-19T20:30:09.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_19.status 2026-02-19T20:30:09.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_19.status 2026-02-19T20:30:09.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:09.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.status 2026-02-19T20:30:09.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_19 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_19.status 2026-02-19T20:30:09.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:09.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:09.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:09.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_19.status 2026-02-19T20:30:09.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_19 --format xml 2026-02-19T20:30:09.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:09.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:09.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=122 2026-02-19T20:30:09.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:09.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:09.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:09.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_19 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.status 2026-02-19T20:30:09.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:09.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:09.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:09.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.status 2026-02-19T20:30:09.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_19 --format xml 2026-02-19T20:30:09.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:10.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:10.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=53 2026-02-19T20:30:10.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 122 = 53 2026-02-19T20:30:10.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:10.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:30:10.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_19 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.status 2026-02-19T20:30:10.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:10.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:10.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:10.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.status 2026-02-19T20:30:10.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_19 --format xml 2026-02-19T20:30:10.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:11.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:11.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:11.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:11.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:11.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:11.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:11.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 122 = '' 2026-02-19T20:30:11.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:11.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:30:11.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_19 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.status 2026-02-19T20:30:11.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:11.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:11.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:11.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.status 2026-02-19T20:30:11.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_19 --format xml 2026-02-19T20:30:11.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:12.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:12.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=122 2026-02-19T20:30:12.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 122 = 122 2026-02-19T20:30:12.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:12.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_19 snap 2026-02-19T20:30:12.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:12.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:12.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:12.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:12.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:12.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:12.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:13.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_19@snap 2026-02-19T20:30:13.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_19': 2026-02-19T20:30:13.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:13.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:13.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:13.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11df27ee0f5a 2026-02-19T20:30:13.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11df27ee0f5a 2026-02-19T20:30:13.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:13.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:43 2026 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:43 2026 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:43 2026 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 2bb7364a-f57b-450f-8771-0e4854fa0d34 2026-02-19T20:30:13.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:13.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:13.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_19 snap 2026-02-19T20:30:13.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:13.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:30:13.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:13.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:13.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_19.export 2026-02-19T20:30:13.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.export 2026-02-19T20:30:13.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_19.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.export 2026-02-19T20:30:13.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_19@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_19.export 2026-02-19T20:30:13.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:13.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_19@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.export 2026-02-19T20:30:13.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:13.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_19.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.export 2026-02-19T20:30:13.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_19.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_19.export 2026-02-19T20:30:13.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:13.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:13.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_20 2026-02-19T20:30:13.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_20 snap 2026-02-19T20:30:13.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:13.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:13.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:13.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:13.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_20@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:13.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_20 2026-02-19T20:30:13.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:13.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:13.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:13.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_20 started 2026-02-19T20:30:13.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:13.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:13.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:13.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:13.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:13.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:13.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:14.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_20 started 2026-02-19T20:30:14.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:14.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:14.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:14.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:14.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:14.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:14.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_20 2026-02-19T20:30:14.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:14.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:14.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:14.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:14.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:14.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:14.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:14.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:14.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:14.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:14.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:14.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_20 2026-02-19T20:30:14.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:14.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:14.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:14.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:14.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:14.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_20 2026-02-19T20:30:14.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:14.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:14.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:14.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:14.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:14.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:14.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:14.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:14.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_20 2026-02-19T20:30:14.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:14.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:14.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:14.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:14.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:14.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:14.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:14.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_20 2026-02-19T20:30:14.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:14.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:14.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:14.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_20 2026-02-19T20:30:15.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 124 2026-02-19T20:30:15.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_20 2026-02-19T20:30:15.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:15.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:15.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:15.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:15.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:15.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_20.status 2026-02-19T20:30:15.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_20.status 2026-02-19T20:30:15.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:15.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_20.status 2026-02-19T20:30:15.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_20.status 2026-02-19T20:30:15.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_20.status 2026-02-19T20:30:15.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:15.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.status 2026-02-19T20:30:15.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_20 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_20.status 2026-02-19T20:30:15.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:15.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:15.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:15.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_20.status 2026-02-19T20:30:15.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_20 --format xml 2026-02-19T20:30:15.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:15.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:15.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=124 2026-02-19T20:30:15.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:15.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:15.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:15.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_20 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.status 2026-02-19T20:30:15.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:15.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:15.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:15.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.status 2026-02-19T20:30:15.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_20 --format xml 2026-02-19T20:30:15.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:15.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:15.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=54 2026-02-19T20:30:15.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 124 = 54 2026-02-19T20:30:15.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:15.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:30:16.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_20 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.status 2026-02-19T20:30:16.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:16.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:16.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:16.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.status 2026-02-19T20:30:16.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_20 --format xml 2026-02-19T20:30:16.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:16.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:16.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=54 2026-02-19T20:30:16.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 124 = 54 2026-02-19T20:30:16.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:16.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:30:17.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_20 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.status 2026-02-19T20:30:17.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:17.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:17.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:17.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.status 2026-02-19T20:30:17.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_20 --format xml 2026-02-19T20:30:17.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:17.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:17.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:17.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:17.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:17.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:17.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:17.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 124 = '' 2026-02-19T20:30:17.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:17.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:30:18.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_20 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.status 2026-02-19T20:30:18.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:18.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:18.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:18.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.status 2026-02-19T20:30:18.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_20 --format xml 2026-02-19T20:30:18.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:18.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:18.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=124 2026-02-19T20:30:18.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 124 = 124 2026-02-19T20:30:18.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:18.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_20 snap 2026-02-19T20:30:18.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:18.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:18.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:18.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:18.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:18.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:18.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:19.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_20@snap 2026-02-19T20:30:19.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_20': 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123dea99bd94 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123dea99bd94 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:21:56 2026 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:21:56 2026 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:21:56 2026 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 06be59c7-13b7-4ca9-8bf1-ac1f92b09a6e 2026-02-19T20:30:19.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:19.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:19.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_20 snap 2026-02-19T20:30:19.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:19.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:30:19.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:19.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:19.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_20.export 2026-02-19T20:30:19.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.export 2026-02-19T20:30:19.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_20.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.export 2026-02-19T20:30:19.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_20@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_20.export 2026-02-19T20:30:20.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:20.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_20@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.export 2026-02-19T20:30:20.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:20.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_20.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.export 2026-02-19T20:30:20.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_20.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_20.export 2026-02-19T20:30:20.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:20.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:20.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_21 2026-02-19T20:30:20.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_21 snap 2026-02-19T20:30:20.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:20.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:20.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:20.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:20.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_21@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_21 2026-02-19T20:30:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_21 started 2026-02-19T20:30:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:20.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:20.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:20.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:20.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:20.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:20.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:21.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_21 started 2026-02-19T20:30:21.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:21.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:21.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:21.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:21.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:21.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:21.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_21 2026-02-19T20:30:21.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:21.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:21.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:21.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:21.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:21.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:21.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:21.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:21.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_21 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:21.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:21.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:21.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:21.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:21.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:21.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:21.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_21 2026-02-19T20:30:21.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:21.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:21.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:21.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:21.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:21.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:21.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:21.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:21.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_21 2026-02-19T20:30:21.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:21.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:21.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:21.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:21.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_21 2026-02-19T20:30:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_21 2026-02-19T20:30:22.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 126 2026-02-19T20:30:22.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_21 2026-02-19T20:30:22.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:22.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:22.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:22.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:22.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:22.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_21.status 2026-02-19T20:30:22.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_21.status 2026-02-19T20:30:22.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:22.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_21.status 2026-02-19T20:30:22.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_21.status 2026-02-19T20:30:22.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_21.status 2026-02-19T20:30:22.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:22.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.status 2026-02-19T20:30:22.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_21 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_21.status 2026-02-19T20:30:22.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:22.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:22.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:22.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_21.status 2026-02-19T20:30:22.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_21 --format xml 2026-02-19T20:30:22.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:22.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:22.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=126 2026-02-19T20:30:22.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:22.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:22.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:22.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_21 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.status 2026-02-19T20:30:22.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:22.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:22.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:22.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.status 2026-02-19T20:30:22.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_21 --format xml 2026-02-19T20:30:22.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:22.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:22.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:22.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:22.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:22.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:22.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:22.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 126 = '' 2026-02-19T20:30:22.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:22.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:30:23.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_21 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.status 2026-02-19T20:30:23.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:23.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:23.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:23.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.status 2026-02-19T20:30:23.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_21 --format xml 2026-02-19T20:30:23.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:23.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:23.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:23.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:23.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:23.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:23.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:23.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 126 = '' 2026-02-19T20:30:23.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:23.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:30:23.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_21 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.status 2026-02-19T20:30:23.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:23.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:23.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:23.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.status 2026-02-19T20:30:23.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_21 --format xml 2026-02-19T20:30:23.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:23.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=126 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 126 = 126 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_21 snap 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:23.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:24.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_21@snap 2026-02-19T20:30:24.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_21': 2026-02-19T20:30:24.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:24.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:24.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:24.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b840318ddd 2026-02-19T20:30:24.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b840318ddd 2026-02-19T20:30:24.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:24.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:24.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:24.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:24.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:07 2026 2026-02-19T20:30:24.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:07 2026 2026-02-19T20:30:24.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:07 2026 2026-02-19T20:30:24.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:24.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:24.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:24.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 6a7c4609-46c1-4458-b2e2-2fc6d6f26028 2026-02-19T20:30:24.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:24.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_21 snap 2026-02-19T20:30:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:30:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:24.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:24.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_21.export 2026-02-19T20:30:24.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.export 2026-02-19T20:30:24.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_21.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.export 2026-02-19T20:30:24.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_21@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_21.export 2026-02-19T20:30:25.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:25.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_21@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.export 2026-02-19T20:30:25.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:25.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_21.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.export 2026-02-19T20:30:25.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_21.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_21.export 2026-02-19T20:30:25.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:25.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:25.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_22 2026-02-19T20:30:25.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_22 snap 2026-02-19T20:30:25.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:25.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:25.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:25.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:25.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_22@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:25.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_22 2026-02-19T20:30:25.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_22 started 2026-02-19T20:30:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:25.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:25.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:25.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:25.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:26.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_22 started 2026-02-19T20:30:26.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:26.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:26.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:26.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:26.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:26.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:26.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_22 2026-02-19T20:30:26.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:26.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:26.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:26.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:26.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:26.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:26.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:26.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:26.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:26.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_22 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:26.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:26.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:26.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:26.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:26.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:26.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:26.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:26.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:26.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:26.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:26.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:26.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_22 2026-02-19T20:30:26.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_22'] 2026-02-19T20:30:26.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:26.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_22 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:30:26.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:30:26.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:30:26.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:26.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:26.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:26.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_22 2026-02-19T20:30:26.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_22'] 2026-02-19T20:30:26.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:26.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:26.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:26.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_22 2026-02-19T20:30:26.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:30:26.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:26.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:26.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:30:26.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:30:26.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:26.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:26.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:26.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:30:26.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:30:26.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:26.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:26.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:30:26.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:30:26.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:30:26.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:30:26.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:26.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:26.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:30:26.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:30:26.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:26.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:26.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:26.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:30:26.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:30:26.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:26.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:26.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:30:26.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:30:26.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:30:26.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:30:26.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:30:26.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:30:26.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_22 2026-02-19T20:30:26.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_22'] 2026-02-19T20:30:26.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:26.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_22 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:26.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:26.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:30:26.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:30:26.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:26.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:26.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:30:26.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:30:26.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:30:26.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:30:26.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:30:26.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:30:26.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_22 2026-02-19T20:30:26.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:26.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:26.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:26.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:26.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:26.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:26.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:26.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:26.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_22 2026-02-19T20:30:26.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:26.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:26.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:26.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:26.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:26.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:26.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:26.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_22 2026-02-19T20:30:26.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:26.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:26.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:26.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_22 2026-02-19T20:30:27.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 128 2026-02-19T20:30:27.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_22 2026-02-19T20:30:27.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:27.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:27.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:27.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:27.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:27.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_22.status 2026-02-19T20:30:27.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_22.status 2026-02-19T20:30:27.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:27.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_22.status 2026-02-19T20:30:27.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_22.status 2026-02-19T20:30:27.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_22.status 2026-02-19T20:30:27.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:27.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.status 2026-02-19T20:30:27.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_22 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_22.status 2026-02-19T20:30:27.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:27.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:27.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:27.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_22.status 2026-02-19T20:30:27.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_22 --format xml 2026-02-19T20:30:27.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:27.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:27.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=128 2026-02-19T20:30:27.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:27.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:27.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:27.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_22 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.status 2026-02-19T20:30:27.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:27.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:27.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:27.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.status 2026-02-19T20:30:27.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_22 --format xml 2026-02-19T20:30:27.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:27.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:27.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=56 2026-02-19T20:30:27.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 128 = 56 2026-02-19T20:30:27.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:27.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:30:28.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_22 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.status 2026-02-19T20:30:28.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:28.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:28.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:28.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.status 2026-02-19T20:30:28.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_22 --format xml 2026-02-19T20:30:28.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:28.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:28.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=56 2026-02-19T20:30:28.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 128 = 56 2026-02-19T20:30:28.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:28.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:30:29.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_22 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.status 2026-02-19T20:30:29.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:29.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:29.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:29.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.status 2026-02-19T20:30:29.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_22 --format xml 2026-02-19T20:30:29.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:29.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:29.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:29.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:29.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:29.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:29.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:29.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 128 = '' 2026-02-19T20:30:29.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:29.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:30:30.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_22 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.status 2026-02-19T20:30:30.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:30.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:30.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:30.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.status 2026-02-19T20:30:30.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_22 --format xml 2026-02-19T20:30:30.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:30.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=128 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 128 = 128 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_22 snap 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:30.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:31.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_22@snap 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_22': 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113acf37bdb7 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113acf37bdb7 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:16 2026 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:16 2026 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:16 2026 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: f9647a4e-5b33-4f6e-bf71-dcbe6192f2bc 2026-02-19T20:30:31.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:31.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:31.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_22 snap 2026-02-19T20:30:31.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:31.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:30:31.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:31.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:31.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_22.export 2026-02-19T20:30:31.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.export 2026-02-19T20:30:31.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_22.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.export 2026-02-19T20:30:31.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_22@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_22.export 2026-02-19T20:30:31.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:31.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_22@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.export 2026-02-19T20:30:32.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:32.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_22.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.export 2026-02-19T20:30:32.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_22.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_22.export 2026-02-19T20:30:32.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:32.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:32.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_23 2026-02-19T20:30:32.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_23 snap 2026-02-19T20:30:32.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:32.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:32.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:32.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:32.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_23@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:33.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_23 2026-02-19T20:30:33.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:33.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:33.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:33.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_23 started 2026-02-19T20:30:33.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:33.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:33.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:33.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:33.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:33.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:33.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:34.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_23 started 2026-02-19T20:30:34.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_23 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:34.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:34.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:34.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:34.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:34.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:34.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_23 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:34.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:34.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:34.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:34.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:34.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:34.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_23 2026-02-19T20:30:34.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_23'] 2026-02-19T20:30:34.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:34.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:34.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:34.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_23 2026-02-19T20:30:34.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:30:34.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:34.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:34.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:30:34.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:30:34.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:34.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:30:34.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:30:34.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:34.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:34.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:34.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_23 2026-02-19T20:30:34.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:34.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:34.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:34.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:34.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:34.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_23 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:34.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_23 2026-02-19T20:30:34.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:34.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:34.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:34.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_23 2026-02-19T20:30:35.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 130 2026-02-19T20:30:35.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_23 2026-02-19T20:30:35.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:35.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:35.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:35.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:35.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:35.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_23.status 2026-02-19T20:30:35.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_23.status 2026-02-19T20:30:35.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:35.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_23.status 2026-02-19T20:30:35.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_23.status 2026-02-19T20:30:35.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_23.status 2026-02-19T20:30:35.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:35.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.status 2026-02-19T20:30:35.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_23 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_23.status 2026-02-19T20:30:35.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:35.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:35.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:35.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_23.status 2026-02-19T20:30:35.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_23 --format xml 2026-02-19T20:30:35.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:35.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:35.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=130 2026-02-19T20:30:35.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:35.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:35.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:35.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_23 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.status 2026-02-19T20:30:35.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:35.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:35.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:35.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.status 2026-02-19T20:30:35.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_23 --format xml 2026-02-19T20:30:35.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:35.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:35.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=57 2026-02-19T20:30:35.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 130 = 57 2026-02-19T20:30:35.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:35.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:30:36.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_23 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.status 2026-02-19T20:30:36.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:36.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:36.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:36.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.status 2026-02-19T20:30:36.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_23 --format xml 2026-02-19T20:30:36.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:36.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:36.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=57 2026-02-19T20:30:36.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 130 = 57 2026-02-19T20:30:36.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:36.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:30:36.841 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5982941121696943) > weight (0.25) 2026-02-19T20:30:36.841 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6352861754343809) > weight (0.25) 2026-02-19T20:30:36.841 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.867427810970719) > weight (0.25) 2026-02-19T20:30:36.841 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.630912653122092) > weight (0.25) 2026-02-19T20:30:36.841 INFO:tasks.rbd_mirror_thrash:waiting for 62.05510324952482 secs before thrashing 2026-02-19T20:30:37.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_23 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.status 2026-02-19T20:30:37.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:37.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:37.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:37.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.status 2026-02-19T20:30:37.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_23 --format xml 2026-02-19T20:30:37.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:37.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:37.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:37.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:37.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:37.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:37.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:37.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 130 = '' 2026-02-19T20:30:37.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:37.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:30:38.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_23 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.status 2026-02-19T20:30:38.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:38.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:38.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:38.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.status 2026-02-19T20:30:38.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_23 --format xml 2026-02-19T20:30:38.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:38.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:38.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=130 2026-02-19T20:30:38.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 130 = 130 2026-02-19T20:30:38.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:38.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_23 snap 2026-02-19T20:30:38.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:38.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:38.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:38.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:38.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:38.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:38.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:39.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_23@snap 2026-02-19T20:30:39.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_23': 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11dfbedf5a98 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11dfbedf5a98 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:29 2026 2026-02-19T20:30:39.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:29 2026 2026-02-19T20:30:39.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:29 2026 2026-02-19T20:30:39.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:39.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:39.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:39.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 8b84cd27-943f-470d-8e23-786f49d70f13 2026-02-19T20:30:39.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:39.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:39.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_23 snap 2026-02-19T20:30:39.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:39.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:30:39.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:39.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:39.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_23.export 2026-02-19T20:30:39.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.export 2026-02-19T20:30:39.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_23.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.export 2026-02-19T20:30:39.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_23@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_23.export 2026-02-19T20:30:39.998 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:40.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_23@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.export 2026-02-19T20:30:40.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:40.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_23.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.export 2026-02-19T20:30:40.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_23.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_23.export 2026-02-19T20:30:40.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:40.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:40.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_24 2026-02-19T20:30:40.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_24 snap 2026-02-19T20:30:40.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:40.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:40.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:40.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:40.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_24@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:40.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_24 2026-02-19T20:30:40.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:40.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_24 started 2026-02-19T20:30:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:40.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:41.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_24 started 2026-02-19T20:30:41.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:41.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:41.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:41.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:41.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:41.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:41.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_24 2026-02-19T20:30:41.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:41.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:41.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:41.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:41.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:41.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:41.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:41.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:41.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:41.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:41.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:41.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_24 2026-02-19T20:30:41.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:41.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:41.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:41.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:41.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:41.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:41.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:41.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:41.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:41.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:41.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:41.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_24 2026-02-19T20:30:41.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:41.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:41.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:41.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:41.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:41.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:41.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:41.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:41.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_24 2026-02-19T20:30:41.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:41.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:41.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:41.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:41.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:41.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:41.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:41.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_24 2026-02-19T20:30:41.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:41.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:41.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:41.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_24 2026-02-19T20:30:42.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 132 2026-02-19T20:30:42.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_24 2026-02-19T20:30:42.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:42.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:42.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:42.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:42.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:42.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_24.status 2026-02-19T20:30:42.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_24.status 2026-02-19T20:30:42.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:42.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_24.status 2026-02-19T20:30:42.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_24.status 2026-02-19T20:30:42.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_24.status 2026-02-19T20:30:42.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:42.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.status 2026-02-19T20:30:42.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_24 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_24.status 2026-02-19T20:30:42.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:42.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:42.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:42.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_24.status 2026-02-19T20:30:42.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_24 --format xml 2026-02-19T20:30:42.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:42.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:42.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=132 2026-02-19T20:30:42.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:42.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:42.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:42.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_24 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.status 2026-02-19T20:30:42.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:42.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:42.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:42.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.status 2026-02-19T20:30:42.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_24 --format xml 2026-02-19T20:30:42.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:42.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:42.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=58 2026-02-19T20:30:42.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 132 = 58 2026-02-19T20:30:42.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:42.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:30:42.724 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8736667550287323) > weight (0.25) 2026-02-19T20:30:42.724 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8331864168343814) > weight (0.25) 2026-02-19T20:30:42.724 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.630379382073446) > weight (0.25) 2026-02-19T20:30:42.724 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.38528052128680124) > weight (0.25) 2026-02-19T20:30:42.725 INFO:tasks.rbd_mirror_thrash:waiting for 77.858092109838 secs before thrashing 2026-02-19T20:30:42.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_24 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.status 2026-02-19T20:30:42.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:42.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:42.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:42.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.status 2026-02-19T20:30:42.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_24 --format xml 2026-02-19T20:30:42.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:43.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:43.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:43.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:43.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:43.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:43.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:43.019 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 132 = '' 2026-02-19T20:30:43.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:43.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:30:43.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_24 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.status 2026-02-19T20:30:43.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:43.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:43.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:43.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.status 2026-02-19T20:30:43.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_24 --format xml 2026-02-19T20:30:43.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:43.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:43.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=132 2026-02-19T20:30:43.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 132 = 132 2026-02-19T20:30:43.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:43.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_24 snap 2026-02-19T20:30:43.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:43.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:43.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:43.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:43.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:43.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:43.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:44.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_24@snap 2026-02-19T20:30:44.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_24': 2026-02-19T20:30:44.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:44.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:44.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:44.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123d3422c3a0 2026-02-19T20:30:44.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123d3422c3a0 2026-02-19T20:30:44.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:44.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:44.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:44.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:44.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:41 2026 2026-02-19T20:30:44.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:41 2026 2026-02-19T20:30:44.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:41 2026 2026-02-19T20:30:44.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:44.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:44.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:44.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 79b55a6a-adb4-4d23-a92b-d5cbf63850f4 2026-02-19T20:30:44.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:44.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:44.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_24 snap 2026-02-19T20:30:44.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:44.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:30:44.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:44.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:44.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_24.export 2026-02-19T20:30:44.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.export 2026-02-19T20:30:44.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_24.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.export 2026-02-19T20:30:44.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_24@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_24.export 2026-02-19T20:30:45.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:45.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_24@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.export 2026-02-19T20:30:45.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:45.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_24.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.export 2026-02-19T20:30:45.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_24.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_24.export 2026-02-19T20:30:45.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:45.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:45.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_25 2026-02-19T20:30:45.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_25 snap 2026-02-19T20:30:45.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:45.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:45.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:45.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:45.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_25@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_25 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_25 started 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:45.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:45.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:46.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_25 started 2026-02-19T20:30:46.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:46.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:46.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:46.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_25 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:46.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:46.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:46.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:46.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:46.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_25 2026-02-19T20:30:46.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:46.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:46.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:46.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:46.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:46.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:46.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:46.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:46.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:46.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:46.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:46.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:46.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_25 2026-02-19T20:30:46.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:46.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:46.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:46.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:46.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:46.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:46.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:46.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:46.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_25 2026-02-19T20:30:46.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:46.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_25 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:46.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_25 2026-02-19T20:30:47.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 134 2026-02-19T20:30:47.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_25 2026-02-19T20:30:47.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:47.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:47.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:47.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:47.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:47.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_25.status 2026-02-19T20:30:47.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_25.status 2026-02-19T20:30:47.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:47.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_25.status 2026-02-19T20:30:47.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_25.status 2026-02-19T20:30:47.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_25.status 2026-02-19T20:30:47.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:47.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.status 2026-02-19T20:30:47.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_25 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_25.status 2026-02-19T20:30:47.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:47.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:47.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:47.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_25.status 2026-02-19T20:30:47.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_25 --format xml 2026-02-19T20:30:47.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:47.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:47.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=134 2026-02-19T20:30:47.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:47.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:47.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:47.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_25 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.status 2026-02-19T20:30:47.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:47.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:47.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:47.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.status 2026-02-19T20:30:47.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_25 --format xml 2026-02-19T20:30:47.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:47.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:47.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=59 2026-02-19T20:30:47.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 134 = 59 2026-02-19T20:30:47.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:47.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:30:48.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_25 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.status 2026-02-19T20:30:48.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:48.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:48.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:48.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.status 2026-02-19T20:30:48.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_25 --format xml 2026-02-19T20:30:48.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:48.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:48.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=59 2026-02-19T20:30:48.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 134 = 59 2026-02-19T20:30:48.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:48.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:30:49.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_25 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.status 2026-02-19T20:30:49.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:49.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:49.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:49.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.status 2026-02-19T20:30:49.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_25 --format xml 2026-02-19T20:30:49.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:49.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:49.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:49.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:49.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:49.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:49.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:49.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 134 = '' 2026-02-19T20:30:49.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:49.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:30:50.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_25 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.status 2026-02-19T20:30:50.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:50.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:50.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:50.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.status 2026-02-19T20:30:50.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_25 --format xml 2026-02-19T20:30:50.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:50.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:50.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=134 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 134 = 134 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_25 snap 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:50.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:51.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_25@snap 2026-02-19T20:30:51.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_25': 2026-02-19T20:30:51.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:51.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:51.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:51.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b89711c56 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b89711c56 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:22:49 2026 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:22:49 2026 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:22:49 2026 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: de88305c-0e9b-4c57-856e-bc3028b13752 2026-02-19T20:30:51.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:51.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:51.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_25 snap 2026-02-19T20:30:51.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:51.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:30:51.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:51.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:51.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_25.export 2026-02-19T20:30:51.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.export 2026-02-19T20:30:51.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_25.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.export 2026-02-19T20:30:51.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_25@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_25.export 2026-02-19T20:30:52.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:52.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_25@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.export 2026-02-19T20:30:52.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:52.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_25.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.export 2026-02-19T20:30:52.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_25.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_25.export 2026-02-19T20:30:52.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:52.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:52.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_26 2026-02-19T20:30:52.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_26 snap 2026-02-19T20:30:52.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:52.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:52.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:52.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:52.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_26@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:52.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_26 2026-02-19T20:30:52.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_26 started 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:52.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:53.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_26 started 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_26 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:53.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:53.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:53.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:53.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_26 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:53.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:53.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:53.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:53.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:53.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:53.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_26 2026-02-19T20:30:53.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_26'] 2026-02-19T20:30:53.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:53.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_26 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:30:53.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:30:53.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:30:53.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:53.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:53.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:53.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_26 2026-02-19T20:30:53.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_26'] 2026-02-19T20:30:53.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:53.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:53.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_26 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:53.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:30:53.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:30:53.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:30:53.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:30:53.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:30:53.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:30:53.479 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_26 2026-02-19T20:30:53.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_26'] 2026-02-19T20:30:53.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:53.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:53.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_26 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:30:53.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:30:53.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:30:53.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:30:53.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:30:53.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:30:53.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_26 2026-02-19T20:30:53.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:53.583 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:53.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:53.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_26 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:53.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:53.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:53.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:53.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_26 2026-02-19T20:30:53.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:53.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:53.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:53.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_26 2026-02-19T20:30:54.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 136 2026-02-19T20:30:54.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_26 2026-02-19T20:30:54.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:54.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:54.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:54.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:54.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:54.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_26.status 2026-02-19T20:30:54.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_26.status 2026-02-19T20:30:54.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:54.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_26.status 2026-02-19T20:30:54.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_26.status 2026-02-19T20:30:54.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_26.status 2026-02-19T20:30:54.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:54.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.status 2026-02-19T20:30:54.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_26 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_26.status 2026-02-19T20:30:54.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:54.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:54.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:54.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_26.status 2026-02-19T20:30:54.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_26 --format xml 2026-02-19T20:30:54.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:54.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:54.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=136 2026-02-19T20:30:54.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:54.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:54.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_26 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.status 2026-02-19T20:30:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:54.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.status 2026-02-19T20:30:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_26 --format xml 2026-02-19T20:30:54.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:54.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:54.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=60 2026-02-19T20:30:54.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 136 = 60 2026-02-19T20:30:54.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:54.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:30:55.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_26 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.status 2026-02-19T20:30:55.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:55.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:55.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:55.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.status 2026-02-19T20:30:55.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_26 --format xml 2026-02-19T20:30:55.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:55.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:55.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:55.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:30:55.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:30:55.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:30:55.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:30:55.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 136 = '' 2026-02-19T20:30:55.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:55.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:30:55.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_26 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.status 2026-02-19T20:30:55.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:55.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:55.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:55.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.status 2026-02-19T20:30:55.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_26 --format xml 2026-02-19T20:30:55.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:55.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:55.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=136 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 136 = 136 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_26 snap 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:30:55.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:56.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_26@snap 2026-02-19T20:30:56.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_26': 2026-02-19T20:30:56.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:30:56.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:30:56.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113a71cabf34 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113a71cabf34 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:01 2026 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:01 2026 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:01 2026 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:30:56.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:30:56.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 807691bb-e01b-4205-9b22-0a3c88b5aa24 2026-02-19T20:30:56.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:30:56.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:56.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_26 snap 2026-02-19T20:30:56.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:56.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:30:56.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:56.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:30:56.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_26.export 2026-02-19T20:30:56.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.export 2026-02-19T20:30:56.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_26.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.export 2026-02-19T20:30:56.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_26@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_26.export 2026-02-19T20:30:57.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:57.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_26@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.export 2026-02-19T20:30:57.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:30:57.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_26.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.export 2026-02-19T20:30:57.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_26.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_26.export 2026-02-19T20:30:57.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:57.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:30:57.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_27 2026-02-19T20:30:57.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_27 snap 2026-02-19T20:30:57.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:57.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:57.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:30:57.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:30:57.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_27@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:30:57.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_27 2026-02-19T20:30:57.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:57.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:57.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:30:57.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_27 started 2026-02-19T20:30:57.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:57.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:57.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:30:57.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:30:57.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:30:57.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:57.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:30:58.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_27 started 2026-02-19T20:30:58.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:58.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:58.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:30:58.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:30:58.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:30:58.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:30:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_27 2026-02-19T20:30:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:30:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:30:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:30:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:30:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:30:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:30:58.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:30:58.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:30:58.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_27 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:58.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:30:58.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:30:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:30:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:30:58.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:30:58.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:58.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:58.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:30:58.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_27 2026-02-19T20:30:58.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_27'] 2026-02-19T20:30:58.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:30:58.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_27 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:30:58.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:30:58.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:30:58.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:30:58.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:30:58.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:30:58.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:30:58.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:30:58.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:30:58.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:30:58.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:30:58.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:58.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:58.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:30:58.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_27 2026-02-19T20:30:58.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:30:58.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:30:58.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:30:58.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:30:58.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_27 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:30:58.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:30:58.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:30:58.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_27 2026-02-19T20:30:58.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:58.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:58.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:30:58.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_27 2026-02-19T20:30:59.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 138 2026-02-19T20:30:59.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_27 2026-02-19T20:30:59.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:30:59.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:59.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:30:59.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:30:59.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:30:59.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_27.status 2026-02-19T20:30:59.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_27.status 2026-02-19T20:30:59.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:59.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_27.status 2026-02-19T20:30:59.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_27.status 2026-02-19T20:30:59.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_27.status 2026-02-19T20:30:59.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:30:59.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.status 2026-02-19T20:30:59.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_27 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_27.status 2026-02-19T20:30:59.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:30:59.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:59.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:30:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_27.status 2026-02-19T20:30:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_27 --format xml 2026-02-19T20:30:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:59.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:30:59.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=138 2026-02-19T20:30:59.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:30:59.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:59.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:30:59.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_27 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.status 2026-02-19T20:30:59.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:30:59.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:30:59.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:30:59.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.status 2026-02-19T20:30:59.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_27 --format xml 2026-02-19T20:30:59.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:30:59.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:30:59.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=61 2026-02-19T20:30:59.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 138 = 61 2026-02-19T20:30:59.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:30:59.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:00.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_27 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.status 2026-02-19T20:31:00.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:00.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:00.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:31:00.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.status 2026-02-19T20:31:00.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_27 --format xml 2026-02-19T20:31:00.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:00.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:00.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=61 2026-02-19T20:31:00.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 138 = 61 2026-02-19T20:31:00.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:00.375 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:01.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_27 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.status 2026-02-19T20:31:01.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:01.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:01.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:31:01.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.status 2026-02-19T20:31:01.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_27 --format xml 2026-02-19T20:31:01.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:01.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:01.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:01.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:01.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:01.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:01.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:01.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 138 = '' 2026-02-19T20:31:01.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:01.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:31:02.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_27 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.status 2026-02-19T20:31:02.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:02.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:02.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:31:02.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.status 2026-02-19T20:31:02.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_27 --format xml 2026-02-19T20:31:02.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:02.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:02.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=138 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 138 = 138 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_27 snap 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:02.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:03.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_27@snap 2026-02-19T20:31:03.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_27': 2026-02-19T20:31:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11dfe582aafd 2026-02-19T20:31:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11dfe582aafd 2026-02-19T20:31:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:03.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:03.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:07 2026 2026-02-19T20:31:03.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:07 2026 2026-02-19T20:31:03.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:07 2026 2026-02-19T20:31:03.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:03.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:03.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:03.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 26aa151c-5c6e-407f-84a4-06e391048d6e 2026-02-19T20:31:03.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:03.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:03.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_27 snap 2026-02-19T20:31:03.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:03.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:31:03.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:03.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:03.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_27.export 2026-02-19T20:31:03.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.export 2026-02-19T20:31:03.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_27.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.export 2026-02-19T20:31:03.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_27@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_27.export 2026-02-19T20:31:03.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:04.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_27@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.export 2026-02-19T20:31:04.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:04.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_27.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.export 2026-02-19T20:31:04.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_27.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_27.export 2026-02-19T20:31:04.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:04.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:04.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_28 2026-02-19T20:31:04.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_28 snap 2026-02-19T20:31:04.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:04.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:04.236 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:04.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:04.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_28@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:04.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_28 2026-02-19T20:31:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_28 started 2026-02-19T20:31:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:04.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:04.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:04.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:05.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_28 started 2026-02-19T20:31:05.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:05.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:05.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:05.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:31:05.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:31:05.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:31:05.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_28 2026-02-19T20:31:05.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:31:05.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:31:05.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:05.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:31:05.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:31:05.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:31:05.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:05.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:31:05.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:31:05.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:31:05.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:05.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_28 2026-02-19T20:31:05.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:31:05.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:05.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:05.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:31:05.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:31:05.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:05.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:05.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:05.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:31:05.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:05.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:05.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:05.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:31:05.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:05.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:31:05.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:31:05.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:05.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:05.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:31:05.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:31:05.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:05.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:05.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:05.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:05.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:05.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:05.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:05.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:31:05.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:31:05.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:31:05.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:05.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:05.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:05.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_28 2026-02-19T20:31:05.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:31:05.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:31:05.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:31:05.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:31:05.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:31:05.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:31:05.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_28 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_28 2026-02-19T20:31:05.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:05.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:05.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:05.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_28 2026-02-19T20:31:06.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 140 2026-02-19T20:31:06.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_28 2026-02-19T20:31:06.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:06.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:06.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:06.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:06.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:06.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_28.status 2026-02-19T20:31:06.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_28.status 2026-02-19T20:31:06.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:06.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_28.status 2026-02-19T20:31:06.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_28.status 2026-02-19T20:31:06.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_28.status 2026-02-19T20:31:06.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:06.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.status 2026-02-19T20:31:06.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_28 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_28.status 2026-02-19T20:31:06.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:06.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:06.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:06.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_28.status 2026-02-19T20:31:06.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_28 --format xml 2026-02-19T20:31:06.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:06.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:31:06.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=140 2026-02-19T20:31:06.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:06.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:06.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:31:06.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_28 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.status 2026-02-19T20:31:06.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:06.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:06.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:06.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.status 2026-02-19T20:31:06.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_28 --format xml 2026-02-19T20:31:06.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:06.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:06.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=62 2026-02-19T20:31:06.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 140 = 62 2026-02-19T20:31:06.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:06.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:07.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_28 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.status 2026-02-19T20:31:07.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:07.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:07.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:07.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.status 2026-02-19T20:31:07.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_28 --format xml 2026-02-19T20:31:07.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:07.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:07.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:07.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:07.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:07.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:07.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:07.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 140 = '' 2026-02-19T20:31:07.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:07.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:07.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_28 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.status 2026-02-19T20:31:07.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:07.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:07.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:07.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.status 2026-02-19T20:31:07.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_28 --format xml 2026-02-19T20:31:07.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:07.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:07.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=140 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 140 = 140 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_28 snap 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:07.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:08.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_28@snap 2026-02-19T20:31:08.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_28': 2026-02-19T20:31:08.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:08.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:08.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:08.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123debd38520 2026-02-19T20:31:08.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123debd38520 2026-02-19T20:31:08.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:08.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:08.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:08.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:08.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:20 2026 2026-02-19T20:31:08.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:20 2026 2026-02-19T20:31:08.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:20 2026 2026-02-19T20:31:08.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:08.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:08.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:08.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 0df66c2e-adfc-4e01-b702-4d0a8bd1c719 2026-02-19T20:31:08.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:08.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:08.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_28 snap 2026-02-19T20:31:08.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:08.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:31:08.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:08.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:08.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_28.export 2026-02-19T20:31:08.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.export 2026-02-19T20:31:08.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_28.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.export 2026-02-19T20:31:08.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_28@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_28.export 2026-02-19T20:31:09.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:09.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_28@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.export 2026-02-19T20:31:09.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:09.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_28.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.export 2026-02-19T20:31:09.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_28.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_28.export 2026-02-19T20:31:09.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:09.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:09.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_29 2026-02-19T20:31:09.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_29 snap 2026-02-19T20:31:09.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:09.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:09.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:09.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:09.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_29@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:09.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_29 2026-02-19T20:31:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_29 started 2026-02-19T20:31:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:09.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:09.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:09.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:10.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_29 started 2026-02-19T20:31:10.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:10.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:10.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:10.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:31:10.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:31:10.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:31:10.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_29 2026-02-19T20:31:10.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:31:10.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:31:10.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:10.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:31:10.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:31:10.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:31:10.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:10.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:31:10.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:31:10.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_29 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:31:10.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:10.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:10.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:10.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:10.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:10.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:10.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:10.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:31:10.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:31:10.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:31:10.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:10.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:10.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:10.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_29 2026-02-19T20:31:10.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2026-02-19T20:31:10.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:10.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:10.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:10.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_29 2026-02-19T20:31:10.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:31:10.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:31:10.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:10.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:10.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:10.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:31:10.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:31:10.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:31:10.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:10.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:10.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:10.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_29 2026-02-19T20:31:10.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2026-02-19T20:31:10.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:10.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:10.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_29 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:31:10.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:10.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:10.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:10.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:10.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:10.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:31:10.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:31:10.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:31:10.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:10.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:10.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:10.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_29 2026-02-19T20:31:10.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_29'] 2026-02-19T20:31:10.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:10.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:10.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_29 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:31:10.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:31:10.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:31:10.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:31:10.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:31:10.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_29 2026-02-19T20:31:10.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:31:10.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:31:10.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:31:10.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:31:10.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:31:10.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_29 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_29 2026-02-19T20:31:10.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:10.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:10.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:10.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_29 2026-02-19T20:31:11.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 142 2026-02-19T20:31:11.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_29 2026-02-19T20:31:11.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:11.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:11.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:11.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:11.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:11.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_29.status 2026-02-19T20:31:11.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_29.status 2026-02-19T20:31:11.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:11.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_29.status 2026-02-19T20:31:11.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_29.status 2026-02-19T20:31:11.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_29.status 2026-02-19T20:31:11.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:11.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.status 2026-02-19T20:31:11.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_29 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_29.status 2026-02-19T20:31:11.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:11.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:11.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:11.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_29.status 2026-02-19T20:31:11.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_29 --format xml 2026-02-19T20:31:11.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:11.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:31:11.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=142 2026-02-19T20:31:11.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:11.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:11.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:31:11.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_29 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.status 2026-02-19T20:31:11.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:11.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:11.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:11.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.status 2026-02-19T20:31:11.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_29 --format xml 2026-02-19T20:31:11.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:11.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:11.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=63 2026-02-19T20:31:11.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 142 = 63 2026-02-19T20:31:11.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:11.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:12.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_29 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.status 2026-02-19T20:31:12.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:12.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:12.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:12.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.status 2026-02-19T20:31:12.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_29 --format xml 2026-02-19T20:31:12.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:12.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:12.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=63 2026-02-19T20:31:12.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 142 = 63 2026-02-19T20:31:12.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:12.376 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:13.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_29 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.status 2026-02-19T20:31:13.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:13.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:13.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:13.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.status 2026-02-19T20:31:13.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_29 --format xml 2026-02-19T20:31:13.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:13.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:13.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:13.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:13.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:13.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:13.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:13.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 142 = '' 2026-02-19T20:31:13.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:13.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:31:14.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_29 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.status 2026-02-19T20:31:14.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:14.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:14.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:14.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.status 2026-02-19T20:31:14.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_29 --format xml 2026-02-19T20:31:14.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:14.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:14.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=142 2026-02-19T20:31:14.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 142 = 142 2026-02-19T20:31:14.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:14.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_29 snap 2026-02-19T20:31:14.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:14.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:14.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:14.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:14.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:14.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:14.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:15.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_29@snap 2026-02-19T20:31:15.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_29': 2026-02-19T20:31:15.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 10b8891bc8a7 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.10b8891bc8a7 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:32 2026 2026-02-19T20:31:15.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:32 2026 2026-02-19T20:31:15.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:32 2026 2026-02-19T20:31:15.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:15.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:15.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:15.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 2df60094-cd0e-43df-88af-ea85d6cc9a96 2026-02-19T20:31:15.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:15.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:15.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_29 snap 2026-02-19T20:31:15.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:15.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:31:15.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:15.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:15.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_29.export 2026-02-19T20:31:15.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.export 2026-02-19T20:31:15.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_29.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.export 2026-02-19T20:31:15.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_29@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_29.export 2026-02-19T20:31:15.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:15.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_29@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.export 2026-02-19T20:31:16.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:16.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_29.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.export 2026-02-19T20:31:16.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_29.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_29.export 2026-02-19T20:31:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_30 2026-02-19T20:31:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_30 snap 2026-02-19T20:31:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:16.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:16.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:16.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_30@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:16.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_30 2026-02-19T20:31:16.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:16.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:16.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:16.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_30 started 2026-02-19T20:31:16.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:16.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:16.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:16.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:16.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:16.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:16.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:17.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_30 started 2026-02-19T20:31:17.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:17.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:17.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:17.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_30 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:17.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:31:17.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:31:17.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:31:17.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:17.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_30 2026-02-19T20:31:17.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:31:17.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:31:17.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:17.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:17.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:17.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:31:17.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:31:17.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:31:17.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:17.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:17.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:17.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_30 2026-02-19T20:31:17.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2026-02-19T20:31:17.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:17.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:17.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:17.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_30 2026-02-19T20:31:17.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:17.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:17.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:17.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:17.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:31:17.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:31:17.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:31:17.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:17.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:17.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:17.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_30 2026-02-19T20:31:17.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2026-02-19T20:31:17.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:17.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:17.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_30 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:31:17.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:17.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:17.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:17.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:17.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:17.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:17.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:31:17.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:31:17.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:31:17.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:17.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:17.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:17.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_30 2026-02-19T20:31:17.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_30'] 2026-02-19T20:31:17.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:17.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:17.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:17.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_30 2026-02-19T20:31:17.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:31:17.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:31:17.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:31:17.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:17.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:17.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:31:17.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:31:17.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:31:17.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:31:17.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:31:17.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:31:17.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_30 2026-02-19T20:31:17.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:31:17.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:31:17.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:31:17.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:31:17.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:31:17.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:31:17.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:31:17.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_30 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_30 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:17.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:17.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_30 2026-02-19T20:31:18.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 144 2026-02-19T20:31:18.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_30 2026-02-19T20:31:18.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:18.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:18.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:18.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:18.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:18.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_30.status 2026-02-19T20:31:18.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_30.status 2026-02-19T20:31:18.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:18.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_30.status 2026-02-19T20:31:18.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_30.status 2026-02-19T20:31:18.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_30.status 2026-02-19T20:31:18.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:18.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.status 2026-02-19T20:31:18.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_30 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_30.status 2026-02-19T20:31:18.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:18.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:18.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:18.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_30.status 2026-02-19T20:31:18.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_30 --format xml 2026-02-19T20:31:18.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:18.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:31:18.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=144 2026-02-19T20:31:18.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:18.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:18.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:31:18.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_30 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.status 2026-02-19T20:31:18.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:18.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:18.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:18.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.status 2026-02-19T20:31:18.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_30 --format xml 2026-02-19T20:31:18.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:18.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:18.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=64 2026-02-19T20:31:18.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 144 = 64 2026-02-19T20:31:18.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:18.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:19.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_30 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.status 2026-02-19T20:31:19.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:19.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:19.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:19.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.status 2026-02-19T20:31:19.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_30 --format xml 2026-02-19T20:31:19.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:19.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:19.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=64 2026-02-19T20:31:19.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 144 = 64 2026-02-19T20:31:19.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:19.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:20.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_30 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.status 2026-02-19T20:31:20.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:20.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:20.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:20.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.status 2026-02-19T20:31:20.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_30 --format xml 2026-02-19T20:31:20.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:20.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:20.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:20.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:20.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:20.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:20.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:20.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 144 = '' 2026-02-19T20:31:20.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:20.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:31:21.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_30 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.status 2026-02-19T20:31:21.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:21.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:21.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:21.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.status 2026-02-19T20:31:21.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_30 --format xml 2026-02-19T20:31:21.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:21.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=144 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 144 = 144 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_30 snap 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:21.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:21.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:22.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_30@snap 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_30': 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113afabd4c96 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113afabd4c96 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:42 2026 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:42 2026 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:42 2026 2026-02-19T20:31:22.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:22.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:22.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:22.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 6e1e9027-c0b2-4632-8435-9250281efb2a 2026-02-19T20:31:22.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_30 snap 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_30.export 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.export 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_30.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.export 2026-02-19T20:31:22.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_30@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_30.export 2026-02-19T20:31:23.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:23.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_30@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.export 2026-02-19T20:31:23.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:23.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_30.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.export 2026-02-19T20:31:23.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_30.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_30.export 2026-02-19T20:31:23.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:23.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:23.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_31 2026-02-19T20:31:23.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_31 snap 2026-02-19T20:31:23.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:23.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:23.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:23.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:23.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_31@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:23.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_31 2026-02-19T20:31:23.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:23.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:23.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:23.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_31 started 2026-02-19T20:31:23.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:23.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:23.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:23.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:23.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:23.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:23.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_31 started 2026-02-19T20:31:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:24.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_31 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:31:24.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:24.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:31:24.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:31:24.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_31 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:24.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:24.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:31:24.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:31:24.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:24.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:24.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:24.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_31 2026-02-19T20:31:24.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_31'] 2026-02-19T20:31:24.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:24.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_31 2026-02-19T20:31:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:24.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:24.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:24.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:31:24.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:31:24.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:31:24.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:24.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:24.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:24.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_31 2026-02-19T20:31:24.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:31:24.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:31:24.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:31:24.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:31:24.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:31:24.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:31:24.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:31:24.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:24.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_31 2026-02-19T20:31:24.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:24.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:24.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:24.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:24.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:24.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:31:24.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:31:24.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_31 2026-02-19T20:31:24.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:24.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:24.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:24.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_31 2026-02-19T20:31:25.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 146 2026-02-19T20:31:25.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_31 2026-02-19T20:31:25.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:25.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:25.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:25.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:25.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:25.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_31.status 2026-02-19T20:31:25.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_31.status 2026-02-19T20:31:25.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:25.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_31.status 2026-02-19T20:31:25.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_31.status 2026-02-19T20:31:25.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_31.status 2026-02-19T20:31:25.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:25.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.status 2026-02-19T20:31:25.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_31 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_31.status 2026-02-19T20:31:25.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:25.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:25.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:25.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_31.status 2026-02-19T20:31:25.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_31 --format xml 2026-02-19T20:31:25.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:25.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:31:25.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=146 2026-02-19T20:31:25.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:25.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:25.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:31:25.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_31 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.status 2026-02-19T20:31:25.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:25.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:25.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:25.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.status 2026-02-19T20:31:25.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_31 --format xml 2026-02-19T20:31:25.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:25.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:25.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=65 2026-02-19T20:31:25.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 146 = 65 2026-02-19T20:31:25.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:25.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:26.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_31 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.status 2026-02-19T20:31:26.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:26.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:26.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:26.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.status 2026-02-19T20:31:26.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_31 --format xml 2026-02-19T20:31:26.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:26.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:26.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=65 2026-02-19T20:31:26.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 146 = 65 2026-02-19T20:31:26.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:26.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:27.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_31 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.status 2026-02-19T20:31:27.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:27.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:27.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:27.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.status 2026-02-19T20:31:27.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_31 --format xml 2026-02-19T20:31:27.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:27.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:27.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:27.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:27.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:27.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:27.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:27.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 146 = '' 2026-02-19T20:31:27.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:27.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:31:27.712 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:31:27.708+0000 7f635ae8a640 -1 reset not still connected to 0x6af15f0 2026-02-19T20:31:27.712 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:31:27.708+0000 7f635ae8a640 -1 reset not still connected to 0x7a73860 2026-02-19T20:31:27.712 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:31:27.708+0000 7f635ae8a640 -1 reset not still connected to 0x8860c30 2026-02-19T20:31:27.712 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:31:27.708+0000 7f635ae8a640 -1 reset not still connected to 0x961f040 2026-02-19T20:31:28.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_31 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.status 2026-02-19T20:31:28.817 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:28.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:28.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:28.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.status 2026-02-19T20:31:28.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_31 --format xml 2026-02-19T20:31:28.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:28.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:28.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=146 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 146 = 146 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_31 snap 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:28.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:29.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_31@snap 2026-02-19T20:31:29.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_31': 2026-02-19T20:31:29.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:29.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11dfca612563 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11dfca612563 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:23:52 2026 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:23:52 2026 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:23:52 2026 2026-02-19T20:31:29.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:29.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:29.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:29.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 41ae9cd0-a35e-4cfc-8eb3-abb6b3d0a94b 2026-02-19T20:31:29.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:29.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:29.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_31 snap 2026-02-19T20:31:29.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:29.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:31:29.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:29.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:29.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_31.export 2026-02-19T20:31:29.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.export 2026-02-19T20:31:29.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_31.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.export 2026-02-19T20:31:29.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_31@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_31.export 2026-02-19T20:31:30.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:30.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_31@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.export 2026-02-19T20:31:30.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:30.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_31.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.export 2026-02-19T20:31:30.230 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_31.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_31.export 2026-02-19T20:31:30.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:30.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:30.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_32 2026-02-19T20:31:30.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_32 snap 2026-02-19T20:31:30.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:30.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:30.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:30.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:30.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_32@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:30.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_32 2026-02-19T20:31:30.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:30.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:30.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:30.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_32 started 2026-02-19T20:31:30.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:30.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:30.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:30.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:30.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:30.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:30.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:31.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_32 started 2026-02-19T20:31:31.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:31.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:31.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_32 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:31:31.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:31.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:31:31.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:31:31.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:31:31.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:31.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_32 2026-02-19T20:31:31.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:31:31.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:31.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:31:31.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:31.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:31.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:31:31.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:31:31.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:31:31.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:31.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:31.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:31.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_32 2026-02-19T20:31:31.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_32'] 2026-02-19T20:31:31.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:31.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:31.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:31.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_32 2026-02-19T20:31:31.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:31:31.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:31.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:31.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:31:31.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:31:31.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:31.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:31.417 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:31:31.418 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:31:31.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:31.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:31.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:31.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_32 2026-02-19T20:31:31.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:31:31.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:31:31.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:31:31.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:31:31.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:31:31.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_32 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:31:31.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:31:31.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_32 2026-02-19T20:31:31.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:31.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:31.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:31.474 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_32 2026-02-19T20:31:32.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 148 2026-02-19T20:31:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_32 2026-02-19T20:31:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_32.status 2026-02-19T20:31:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_32.status 2026-02-19T20:31:32.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:32.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_32.status 2026-02-19T20:31:32.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_32.status 2026-02-19T20:31:32.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_32.status 2026-02-19T20:31:32.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:32.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.status 2026-02-19T20:31:32.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_32 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_32.status 2026-02-19T20:31:32.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:32.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:32.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:32.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_32.status 2026-02-19T20:31:32.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_32 --format xml 2026-02-19T20:31:32.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:32.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:31:32.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=148 2026-02-19T20:31:32.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:32.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:32.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:31:32.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_32 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.status 2026-02-19T20:31:32.599 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:32.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:32.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:32.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.status 2026-02-19T20:31:32.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_32 --format xml 2026-02-19T20:31:32.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:32.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:32.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=66 2026-02-19T20:31:32.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 148 = 66 2026-02-19T20:31:32.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:32.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:33.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_32 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.status 2026-02-19T20:31:33.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:33.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:33.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:33.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.status 2026-02-19T20:31:33.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_32 --format xml 2026-02-19T20:31:33.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:33.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:33.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:33.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:33.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:33.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:33.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:33.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 148 = '' 2026-02-19T20:31:33.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:33.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:33.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_32 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.status 2026-02-19T20:31:33.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:33.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:33.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:33.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.status 2026-02-19T20:31:33.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_32 --format xml 2026-02-19T20:31:33.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:33.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:33.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=148 2026-02-19T20:31:33.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 148 = 148 2026-02-19T20:31:33.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:33.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_32 snap 2026-02-19T20:31:33.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:33.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:33.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:33.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:33.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:33.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:33.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:34.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_32@snap 2026-02-19T20:31:34.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_32': 2026-02-19T20:31:34.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 124f66e84874 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.124f66e84874 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:04 2026 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:04 2026 2026-02-19T20:31:34.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:04 2026 2026-02-19T20:31:34.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:34.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:34.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:34.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 8f9355c3-ffc8-4071-aaf4-f8356e2ec162 2026-02-19T20:31:34.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:34.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:34.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_32 snap 2026-02-19T20:31:34.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:34.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:31:34.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:34.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:34.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_32.export 2026-02-19T20:31:34.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.export 2026-02-19T20:31:34.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_32.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.export 2026-02-19T20:31:34.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_32@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_32.export 2026-02-19T20:31:35.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:35.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_32@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.export 2026-02-19T20:31:35.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:35.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_32.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.export 2026-02-19T20:31:35.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_32.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_32.export 2026-02-19T20:31:35.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:35.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:35.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_33 2026-02-19T20:31:35.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_33 snap 2026-02-19T20:31:35.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:35.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:35.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:35.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:35.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_33@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:35.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_33 2026-02-19T20:31:35.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:35.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:35.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:35.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_33 started 2026-02-19T20:31:35.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:35.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:35.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:35.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:35.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:35.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:35.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:36.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_33 started 2026-02-19T20:31:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:31:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:31:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:31:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_33 2026-02-19T20:31:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:31:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:31:36.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:36.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:31:36.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:31:36.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:31:36.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:36.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:31:36.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:31:36.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:31:36.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:36.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_33 2026-02-19T20:31:36.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:31:36.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:36.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:31:36.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:31:36.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:36.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:36.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:31:36.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:31:36.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:36.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:36.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:36.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:36.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:36.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:36.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:36.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:31:36.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:31:36.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:31:36.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:36.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:36.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:36.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_33 2026-02-19T20:31:36.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_33'] 2026-02-19T20:31:36.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:36.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:36.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:36.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_33 2026-02-19T20:31:36.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:31:36.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:36.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:36.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:31:36.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:31:36.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:36.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:36.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:31:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:31:36.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:31:36.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:36.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:36.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:36.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_33 2026-02-19T20:31:36.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_33'] 2026-02-19T20:31:36.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:36.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:36.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_33 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:36.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:31:36.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:31:36.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:31:36.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:36.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:36.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:36.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_33 2026-02-19T20:31:36.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:31:36.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:31:36.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:31:36.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:31:36.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:31:36.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:31:36.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:31:36.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:36.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_33 2026-02-19T20:31:36.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:36.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_33 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:36.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_33 2026-02-19T20:31:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 150 2026-02-19T20:31:37.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_33 2026-02-19T20:31:37.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:37.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:37.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:37.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:37.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:37.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_33.status 2026-02-19T20:31:37.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_33.status 2026-02-19T20:31:37.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:37.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_33.status 2026-02-19T20:31:37.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_33.status 2026-02-19T20:31:37.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_33.status 2026-02-19T20:31:37.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:37.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.status 2026-02-19T20:31:37.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_33 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_33.status 2026-02-19T20:31:37.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:37.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:37.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:37.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_33.status 2026-02-19T20:31:37.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_33 --format xml 2026-02-19T20:31:37.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:37.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:31:37.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=150 2026-02-19T20:31:37.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:37.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:37.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:31:37.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_33 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.status 2026-02-19T20:31:37.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:37.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:37.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:37.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.status 2026-02-19T20:31:37.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_33 --format xml 2026-02-19T20:31:37.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:37.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:37.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:37.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:37.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:37.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:37.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:37.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 150 = '' 2026-02-19T20:31:37.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:37.946 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:38.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_33 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.status 2026-02-19T20:31:38.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:38.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:38.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:38.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.status 2026-02-19T20:31:38.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_33 --format xml 2026-02-19T20:31:38.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:38.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:38.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:38.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:38.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:38.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:38.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:38.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 150 = '' 2026-02-19T20:31:38.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:38.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:38.898 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2026-02-19T20:31:38.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2026-02-19T20:31:38.898 INFO:tasks.rbd_mirror_thrash:waiting for 0.08934339682419856 secs before reviving daemons 2026-02-19T20:31:38.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:31:38.897+0000 7f4fb57cb640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56637) UID: 1000 2026-02-19T20:31:38.906 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:31:38.905+0000 7f5bf91cc640 -1 reset not still connected to 0x1e6f4b60 2026-02-19T20:31:38.988 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2026-02-19T20:31:38.988 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:31:39.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_33 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.status 2026-02-19T20:31:39.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:39.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:39.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:39.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.status 2026-02-19T20:31:39.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_33 --format xml 2026-02-19T20:31:39.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:39.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=150 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 150 = 150 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_33 snap 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:39.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:40.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_33@snap 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_33': 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 124f1509104c 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.124f1509104c 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:16 2026 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:16 2026 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:16 2026 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 4410c659-b877-49f7-b76c-9e1e17ec8cee 2026-02-19T20:31:40.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:40.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:40.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_33 snap 2026-02-19T20:31:40.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:40.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:31:40.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:40.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:40.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_33.export 2026-02-19T20:31:40.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.export 2026-02-19T20:31:40.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_33.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.export 2026-02-19T20:31:40.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_33@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_33.export 2026-02-19T20:31:40.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:40.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_33@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.export 2026-02-19T20:31:40.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:40.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_33.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.export 2026-02-19T20:31:40.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_33.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_33.export 2026-02-19T20:31:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_34 2026-02-19T20:31:40.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_34 snap 2026-02-19T20:31:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:40.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_34@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:40.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_34 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_34 started 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:40.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:40.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:41.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_34 started 2026-02-19T20:31:41.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:41.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:41.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:41.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:31:41.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:31:41.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:31:41.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_34 2026-02-19T20:31:41.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:31:41.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:31:41.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:41.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:31:41.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:31:41.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:31:41.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:41.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:31:41.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:31:41.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:31:41.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:41.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_34 2026-02-19T20:31:41.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:31:41.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:41.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:31:41.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:41.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:41.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:31:41.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:31:41.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:31:41.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:41.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:41.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:41.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_34 2026-02-19T20:31:41.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2026-02-19T20:31:41.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:41.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_34 2026-02-19T20:31:41.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:31:41.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:41.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:41.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:31:41.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:31:41.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:41.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:41.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:41.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:31:41.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:41.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:41.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:41.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:31:41.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:41.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:31:41.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:31:41.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:41.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:41.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:31:41.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:31:41.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:41.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:41.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:41.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:41.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:41.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:41.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:41.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:31:41.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:31:41.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:31:41.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:41.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:41.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:41.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_34 2026-02-19T20:31:41.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2026-02-19T20:31:41.789 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:41.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:41.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:41.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_34 2026-02-19T20:31:41.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:31:41.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:41.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:41.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:31:41.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:41.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:31:41.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:31:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:41.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:41.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_34 2026-02-19T20:31:41.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_34'] 2026-02-19T20:31:41.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:41.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_34 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:41.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:31:41.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:31:41.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:31:41.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:31:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:31:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:31:41.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_34 2026-02-19T20:31:41.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:31:41.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:31:41.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:31:41.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:31:41.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:31:41.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_34 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:31:41.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_34 2026-02-19T20:31:41.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:41.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:41.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:41.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_34 2026-02-19T20:31:42.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 152 2026-02-19T20:31:42.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_34 2026-02-19T20:31:42.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:42.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:42.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:42.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:42.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:42.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_34.status 2026-02-19T20:31:42.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_34.status 2026-02-19T20:31:42.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:42.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_34.status 2026-02-19T20:31:42.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_34.status 2026-02-19T20:31:42.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_34.status 2026-02-19T20:31:42.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:42.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.status 2026-02-19T20:31:42.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_34 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_34.status 2026-02-19T20:31:42.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:42.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:42.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:42.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_34.status 2026-02-19T20:31:42.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_34 --format xml 2026-02-19T20:31:42.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:42.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:31:42.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=152 2026-02-19T20:31:42.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:42.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:42.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:31:42.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_34 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.status 2026-02-19T20:31:42.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:42.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:42.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:42.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.status 2026-02-19T20:31:42.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_34 --format xml 2026-02-19T20:31:42.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:42.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:42.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=68 2026-02-19T20:31:42.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 152 = 68 2026-02-19T20:31:42.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:42.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:43.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_34 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.status 2026-02-19T20:31:43.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:43.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:43.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:43.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.status 2026-02-19T20:31:43.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_34 --format xml 2026-02-19T20:31:43.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:43.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:43.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=68 2026-02-19T20:31:43.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 152 = 68 2026-02-19T20:31:43.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:43.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:44.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_34 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.status 2026-02-19T20:31:44.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:44.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:44.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:44.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.status 2026-02-19T20:31:44.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_34 --format xml 2026-02-19T20:31:44.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:44.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:44.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:44.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:44.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:44.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:44.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:44.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 152 = '' 2026-02-19T20:31:44.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:44.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:31:44.571 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:31:44.569+0000 7f5bf91cc640 -1 reset not still connected to 0x1d57d2b0 2026-02-19T20:31:44.571 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:31:44.569+0000 7f5bf91cc640 -1 reset not still connected to 0x2134dd40 2026-02-19T20:31:45.089 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2026-02-19T20:31:45.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2026-02-19T20:31:45.090 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2026-02-19T20:31:45.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2026-02-19T20:31:45.092 INFO:tasks.rbd_mirror_thrash:waiting for 103.02807837587329 secs before thrashing 2026-02-19T20:31:45.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_34 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.status 2026-02-19T20:31:45.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:45.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:45.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:45.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.status 2026-02-19T20:31:45.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_34 --format xml 2026-02-19T20:31:45.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:45.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:45.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=152 2026-02-19T20:31:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 152 = 152 2026-02-19T20:31:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_34 snap 2026-02-19T20:31:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:45.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:45.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:46.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_34@snap 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_34': 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 124f4bbfac2 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.124f4bbfac2 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:46.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:26 2026 2026-02-19T20:31:46.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:26 2026 2026-02-19T20:31:46.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:26 2026 2026-02-19T20:31:46.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:46.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:46.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:46.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: e3994cd0-fb3f-4a15-bc7e-aa4c9bd26c87 2026-02-19T20:31:46.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_34 snap 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_34.export 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.export 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_34.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.export 2026-02-19T20:31:46.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_34@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_34.export 2026-02-19T20:31:47.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:47.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_34@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.export 2026-02-19T20:31:47.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:47.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_34.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.export 2026-02-19T20:31:47.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_34.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_34.export 2026-02-19T20:31:47.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:47.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:47.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_35 2026-02-19T20:31:47.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_35 snap 2026-02-19T20:31:47.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:47.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:47.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:47.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:47.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_35@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:47.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_35 2026-02-19T20:31:47.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:47.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:47.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:47.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_35 started 2026-02-19T20:31:47.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:47.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:47.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:47.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:47.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:47.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:47.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:48.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_35 started 2026-02-19T20:31:48.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:48.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:48.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:48.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:31:48.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:31:48.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_35 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:31:48.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:31:48.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:31:48.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:48.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_35 2026-02-19T20:31:48.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:31:48.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:48.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:48.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:31:48.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:31:48.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:31:48.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:31:48.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:48.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:48.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:48.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_35 2026-02-19T20:31:48.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:31:48.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:31:48.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:31:48.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:31:48.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:31:48.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:31:48.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:31:48.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:48.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_35 2026-02-19T20:31:48.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_35 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:48.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_35 2026-02-19T20:31:49.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 154 2026-02-19T20:31:49.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_35 2026-02-19T20:31:49.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:49.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:49.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:49.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:49.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:49.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_35.status 2026-02-19T20:31:49.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_35.status 2026-02-19T20:31:49.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:49.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_35.status 2026-02-19T20:31:49.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_35.status 2026-02-19T20:31:49.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_35.status 2026-02-19T20:31:49.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:49.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.status 2026-02-19T20:31:49.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_35 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_35.status 2026-02-19T20:31:49.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:49.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:49.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:49.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_35.status 2026-02-19T20:31:49.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_35 --format xml 2026-02-19T20:31:49.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:49.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:31:49.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=154 2026-02-19T20:31:49.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:49.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:49.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:31:49.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_35 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.status 2026-02-19T20:31:49.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:49.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:49.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:49.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.status 2026-02-19T20:31:49.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_35 --format xml 2026-02-19T20:31:49.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:49.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:49.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=69 2026-02-19T20:31:49.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 154 = 69 2026-02-19T20:31:49.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:49.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:50.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_35 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.status 2026-02-19T20:31:50.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:50.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:50.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:50.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.status 2026-02-19T20:31:50.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_35 --format xml 2026-02-19T20:31:50.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:50.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:50.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=69 2026-02-19T20:31:50.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 154 = 69 2026-02-19T20:31:50.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:50.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:51.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_35 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.status 2026-02-19T20:31:51.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:51.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:51.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:51.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.status 2026-02-19T20:31:51.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_35 --format xml 2026-02-19T20:31:51.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:51.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:51.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:51.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:51.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:51.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:51.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:51.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 154 = '' 2026-02-19T20:31:51.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:51.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:31:52.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_35 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.status 2026-02-19T20:31:52.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:52.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:52.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:52.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.status 2026-02-19T20:31:52.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_35 --format xml 2026-02-19T20:31:52.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:52.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:52.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=154 2026-02-19T20:31:52.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 154 = 154 2026-02-19T20:31:52.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:52.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_35 snap 2026-02-19T20:31:52.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:52.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:52.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:52.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:52.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:52.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:52.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:53.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_35@snap 2026-02-19T20:31:53.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_35': 2026-02-19T20:31:53.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:53.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:53.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:53.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 124fbf67df88 2026-02-19T20:31:53.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.124fbf67df88 2026-02-19T20:31:53.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:53.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:53.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:53.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:33 2026 2026-02-19T20:31:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:33 2026 2026-02-19T20:31:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:33 2026 2026-02-19T20:31:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 1d72d7ff-8dde-42b4-9811-16a6c4bba739 2026-02-19T20:31:53.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:53.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:53.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_35 snap 2026-02-19T20:31:53.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:53.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:31:53.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:53.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:53.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_35.export 2026-02-19T20:31:53.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.export 2026-02-19T20:31:53.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_35.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.export 2026-02-19T20:31:53.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_35@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_35.export 2026-02-19T20:31:54.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:54.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_35@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.export 2026-02-19T20:31:54.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:54.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_35.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.export 2026-02-19T20:31:54.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_35.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_35.export 2026-02-19T20:31:54.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:54.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:54.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_36 2026-02-19T20:31:54.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_36 snap 2026-02-19T20:31:54.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:54.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:54.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:54.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:54.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_36@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:54.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_36 2026-02-19T20:31:54.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:54.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:54.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:54.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_36 started 2026-02-19T20:31:54.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:54.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:54.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:54.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:54.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:54.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:54.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:55.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_36 started 2026-02-19T20:31:55.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:55.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:55.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:55.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:31:55.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:31:55.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:31:55.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_36 2026-02-19T20:31:55.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:31:55.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:31:55.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:55.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:31:55.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:31:55.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:31:55.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:55.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:31:55.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:31:55.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:31:55.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:55.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_36 2026-02-19T20:31:55.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:31:55.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:55.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:55.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:31:55.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:31:55.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:55.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:31:55.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:31:55.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:55.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:55.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:31:55.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_36 2026-02-19T20:31:55.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_36'] 2026-02-19T20:31:55.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:55.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:55.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:55.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_36 2026-02-19T20:31:55.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:31:55.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:55.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:31:55.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:55.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:55.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:31:55.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:31:55.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:31:55.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:55.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:55.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:31:55.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_36 2026-02-19T20:31:55.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_36'] 2026-02-19T20:31:55.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:31:55.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:31:55.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:31:55.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_36 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:31:55.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:31:55.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:31:55.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:31:55.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:31:55.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:31:55.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:31:55.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:31:55.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:31:55.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:31:55.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:31:55.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:55.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:55.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:31:55.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_36 2026-02-19T20:31:55.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:31:55.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:31:55.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:31:55.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:31:55.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:31:55.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:31:55.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_36 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_36 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:55.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:55.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:55.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_36 2026-02-19T20:31:56.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 156 2026-02-19T20:31:56.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_36 2026-02-19T20:31:56.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:31:56.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:56.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:31:56.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:31:56.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:56.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_36.status 2026-02-19T20:31:56.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_36.status 2026-02-19T20:31:56.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:56.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_36.status 2026-02-19T20:31:56.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_36.status 2026-02-19T20:31:56.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_36.status 2026-02-19T20:31:56.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:31:56.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.status 2026-02-19T20:31:56.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_36 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_36.status 2026-02-19T20:31:56.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:56.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:56.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:56.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_36.status 2026-02-19T20:31:56.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_36 --format xml 2026-02-19T20:31:56.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:56.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:31:56.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=156 2026-02-19T20:31:56.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:56.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:56.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:31:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_36 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.status 2026-02-19T20:31:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.status 2026-02-19T20:31:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:56.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_36 --format xml 2026-02-19T20:31:56.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:56.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:56.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:56.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:56.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:56.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:56.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 156 = '' 2026-02-19T20:31:56.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:56.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:31:57.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_36 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.status 2026-02-19T20:31:57.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:57.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:57.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:57.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.status 2026-02-19T20:31:57.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_36 --format xml 2026-02-19T20:31:57.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:57.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:31:57.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:57.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:31:57.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:31:57.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:31:57.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:31:57.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 156 = '' 2026-02-19T20:31:57.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:31:57.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:31:57.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_36 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.status 2026-02-19T20:31:57.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:57.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:57.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:57.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.status 2026-02-19T20:31:57.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_36 --format xml 2026-02-19T20:31:57.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:31:57.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:31:57.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=156 2026-02-19T20:31:57.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 156 = 156 2026-02-19T20:31:57.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:57.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_36 snap 2026-02-19T20:31:57.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:57.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:57.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:57.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:57.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:57.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:31:57.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:31:58.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_36@snap 2026-02-19T20:31:58.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_36': 2026-02-19T20:31:58.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:31:58.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:31:58.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:31:58.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 124f2b09ab37 2026-02-19T20:31:58.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.124f2b09ab37 2026-02-19T20:31:58.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:31:58.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:31:58.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:31:58.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:31:58.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:41 2026 2026-02-19T20:31:58.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:41 2026 2026-02-19T20:31:58.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:41 2026 2026-02-19T20:31:58.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:31:58.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:31:58.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:31:58.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: a86dfcae-7da8-44fb-8bfe-52f9b7766c69 2026-02-19T20:31:58.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:31:58.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:58.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_36 snap 2026-02-19T20:31:58.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:58.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:31:58.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:58.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:31:58.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_36.export 2026-02-19T20:31:58.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.export 2026-02-19T20:31:58.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_36.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.export 2026-02-19T20:31:58.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_36@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_36.export 2026-02-19T20:31:59.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:59.046 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_36@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.export 2026-02-19T20:31:59.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:31:59.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_36.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.export 2026-02-19T20:31:59.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_36.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_36.export 2026-02-19T20:31:59.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:31:59.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:31:59.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_37 2026-02-19T20:31:59.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_37 snap 2026-02-19T20:31:59.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:31:59.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:59.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:31:59.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:31:59.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_37@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:31:59.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_37 2026-02-19T20:31:59.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:59.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:59.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:31:59.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_37 started 2026-02-19T20:31:59.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:31:59.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:31:59.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:31:59.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:31:59.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:31:59.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:31:59.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:00.583 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8134955769493424) > weight (0.25) 2026-02-19T20:32:00.583 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2561269551218376) > weight (0.25) 2026-02-19T20:32:00.583 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8912954833545831) > weight (0.25) 2026-02-19T20:32:00.584 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8506989413754047) > weight (0.25) 2026-02-19T20:32:00.584 INFO:tasks.rbd_mirror_thrash:waiting for 70.56568704416469 secs before thrashing 2026-02-19T20:32:00.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_37 started 2026-02-19T20:32:00.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:00.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_37 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:32:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:00.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:32:00.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_37 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:00.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:00.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:00.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:00.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:00.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:00.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:00.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_37 2026-02-19T20:32:00.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:00.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:00.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:00.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:00.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_37 2026-02-19T20:32:00.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:00.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:00.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:00.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:00.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:00.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:00.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:32:00.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:32:00.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:32:00.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:00.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:00.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:00.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_37 2026-02-19T20:32:00.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:00.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:00.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:00.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:00.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_37 2026-02-19T20:32:00.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:32:00.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:32:00.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:00.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:00.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:00.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:32:00.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:32:00.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:32:00.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:00.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:00.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:00.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_37 2026-02-19T20:32:00.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:00.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:00.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:00.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_37 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:32:00.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:00.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:00.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:00.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:00.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:00.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:00.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:32:00.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:32:00.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:32:00.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:00.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:00.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:00.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_37 2026-02-19T20:32:00.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:00.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:00.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:00.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:00.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 1 2026-02-19T20:32:01.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:01.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:01.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:01.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_37 2026-02-19T20:32:01.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:01.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:01.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:01.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:01.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:01.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:01.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:01.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:01.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:01.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:01.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:01.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_37 2026-02-19T20:32:01.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:01.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:01.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:01.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_37 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:32:01.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:01.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:01.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:01.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:01.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:01.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:01.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:01.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:32:01.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:32:01.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:32:01.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:01.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:01.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:01.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_37 2026-02-19T20:32:02.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:02.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:02.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:02.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:02.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_37 2026-02-19T20:32:02.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:32:02.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:02.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:02.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:32:02.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:32:02.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:02.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:32:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:32:02.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:32:02.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:02.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:02.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:02.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_37 2026-02-19T20:32:02.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:02.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:02.060 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_37 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:02.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:32:02.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:32:02.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:32:02.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:02.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:02.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:02.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_37 2026-02-19T20:32:02.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:02.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:02.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:02.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:02.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 2 2026-02-19T20:32:04.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:04.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:04.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:04.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_37 2026-02-19T20:32:04.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:04.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:04.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:04.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:04.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:04.123 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:04.124 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:04.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:04.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:04.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:04.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:04.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:04.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:04.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_37 2026-02-19T20:32:04.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:04.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:04.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:04.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:04.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_37 2026-02-19T20:32:04.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:04.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:04.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:04.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:04.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:04.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:04.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:32:04.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:32:04.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:32:04.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:04.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:04.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:04.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_37 2026-02-19T20:32:04.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:04.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:04.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:04.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_37 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:04.232 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:32:04.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:32:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:32:04.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:32:04.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:04.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:04.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:04.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_37 2026-02-19T20:32:04.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:04.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:04.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:04.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:04.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_37 2026-02-19T20:32:04.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:04.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:04.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:04.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:04.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:04.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:32:04.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:32:04.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:32:04.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:04.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:04.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:04.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_37 2026-02-19T20:32:04.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:04.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:04.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:04.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:04.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 4 2026-02-19T20:32:08.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:08.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:08.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:08.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_37 2026-02-19T20:32:08.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:08.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:08.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:08.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:08.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:08.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:08.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:08.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:08.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:08.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:08.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:08.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:08.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:08.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:08.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_37 2026-02-19T20:32:08.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:08.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:08.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_37 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:08.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:32:08.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:32:08.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:32:08.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:08.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:08.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:08.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_37 2026-02-19T20:32:08.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:08.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:08.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_37 2026-02-19T20:32:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:32:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:32:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:32:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:08.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:08.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:32:08.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:32:08.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:08.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:08.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:08.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_37 2026-02-19T20:32:08.506 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_37'] 2026-02-19T20:32:08.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:08.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:08.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:08.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_37 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:32:08.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:32:08.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:32:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:32:08.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:32:08.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:08.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:08.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:08.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_37 2026-02-19T20:32:08.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:32:08.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:32:08.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:32:08.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:32:08.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:32:08.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_37 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:32:08.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_37 2026-02-19T20:32:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:08.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_37 2026-02-19T20:32:08.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 158 2026-02-19T20:32:08.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_37 2026-02-19T20:32:08.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:08.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:08.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:08.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:08.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:08.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_37.status 2026-02-19T20:32:08.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_37.status 2026-02-19T20:32:08.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:08.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_37.status 2026-02-19T20:32:08.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_37.status 2026-02-19T20:32:08.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_37.status 2026-02-19T20:32:08.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:08.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.status 2026-02-19T20:32:08.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_37 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_37.status 2026-02-19T20:32:08.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:08.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:08.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:08.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_37.status 2026-02-19T20:32:08.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_37 --format xml 2026-02-19T20:32:08.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:08.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:32:08.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=158 2026-02-19T20:32:08.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:08.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:08.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:32:08.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_37 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.status 2026-02-19T20:32:08.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:08.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:08.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:08.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.status 2026-02-19T20:32:08.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_37 --format xml 2026-02-19T20:32:08.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:08.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:08.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=71 2026-02-19T20:32:08.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 158 = 71 2026-02-19T20:32:08.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:08.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:32:09.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_37 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.status 2026-02-19T20:32:09.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:09.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:09.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:09.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.status 2026-02-19T20:32:09.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_37 --format xml 2026-02-19T20:32:09.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:09.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:09.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=71 2026-02-19T20:32:09.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 158 = 71 2026-02-19T20:32:09.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:09.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:32:10.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_37 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.status 2026-02-19T20:32:10.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:10.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.status 2026-02-19T20:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_37 --format xml 2026-02-19T20:32:10.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:10.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:10.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:10.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:10.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 158 = '' 2026-02-19T20:32:10.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:10.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:32:11.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_37 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.status 2026-02-19T20:32:11.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:11.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:11.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:11.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.status 2026-02-19T20:32:11.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_37 --format xml 2026-02-19T20:32:11.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:11.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=158 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 158 = 158 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_37 snap 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:32:11.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:12.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_37@snap 2026-02-19T20:32:12.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_37': 2026-02-19T20:32:12.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:32:12.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:32:12.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:32:12.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 124fcd796e70 2026-02-19T20:32:12.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.124fcd796e70 2026-02-19T20:32:12.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:32:12.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:32:12.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:32:12.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:32:12.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:24:52 2026 2026-02-19T20:32:12.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:24:52 2026 2026-02-19T20:32:12.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:24:52 2026 2026-02-19T20:32:12.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:32:12.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:32:12.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:32:12.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: a1238f9d-e6e7-407f-88bc-b2822ea2d0df 2026-02-19T20:32:12.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_37 snap 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_37.export 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.export 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_37.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.export 2026-02-19T20:32:12.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_37@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_37.export 2026-02-19T20:32:12.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:12.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_37@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.export 2026-02-19T20:32:13.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:13.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_37.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.export 2026-02-19T20:32:13.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_37.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_37.export 2026-02-19T20:32:13.191 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:13.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:32:13.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_38 2026-02-19T20:32:13.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_38 snap 2026-02-19T20:32:13.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:13.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:13.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:13.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:13.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_38@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:32:13.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_38 2026-02-19T20:32:13.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_38 started 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:13.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:14.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_38 started 2026-02-19T20:32:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:32:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:32:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:32:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_38 2026-02-19T20:32:14.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:32:14.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:32:14.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:14.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:32:14.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:32:14.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:32:14.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:14.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:32:14.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:14.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:14.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_38 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:14.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:14.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:14.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:14.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:14.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:14.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:14.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_38 2026-02-19T20:32:14.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_38'] 2026-02-19T20:32:14.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:14.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:14.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_38 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:32:14.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:14.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:14.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:14.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:14.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:14.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:14.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:14.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:32:14.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:32:14.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:32:14.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:14.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:14.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:14.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_38 2026-02-19T20:32:14.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:32:14.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:32:14.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:32:14.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:32:14.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:32:14.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:32:14.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:32:14.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:14.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_38 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_38 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:14.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_38 2026-02-19T20:32:15.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 160 2026-02-19T20:32:15.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_38 2026-02-19T20:32:15.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:15.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:15.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:15.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:15.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:15.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_38.status 2026-02-19T20:32:15.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_38.status 2026-02-19T20:32:15.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:15.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_38.status 2026-02-19T20:32:15.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_38.status 2026-02-19T20:32:15.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_38.status 2026-02-19T20:32:15.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:15.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.status 2026-02-19T20:32:15.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_38 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_38.status 2026-02-19T20:32:15.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:15.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:15.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:15.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_38.status 2026-02-19T20:32:15.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_38 --format xml 2026-02-19T20:32:15.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:15.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:32:15.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=160 2026-02-19T20:32:15.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:15.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:15.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:32:15.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_38 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.status 2026-02-19T20:32:15.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:15.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:15.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:15.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.status 2026-02-19T20:32:15.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_38 --format xml 2026-02-19T20:32:15.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:15.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:15.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=72 2026-02-19T20:32:15.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 160 = 72 2026-02-19T20:32:15.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:15.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:32:16.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_38 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.status 2026-02-19T20:32:16.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:16.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:16.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:16.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.status 2026-02-19T20:32:16.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_38 --format xml 2026-02-19T20:32:16.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:16.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:16.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=72 2026-02-19T20:32:16.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 160 = 72 2026-02-19T20:32:16.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:16.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:32:17.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_38 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.status 2026-02-19T20:32:17.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:17.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:17.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.status 2026-02-19T20:32:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_38 --format xml 2026-02-19T20:32:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:17.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:17.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:17.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:17.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:17.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:17.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:17.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 160 = '' 2026-02-19T20:32:17.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:17.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:32:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_38 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.status 2026-02-19T20:32:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:18.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.status 2026-02-19T20:32:18.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_38 --format xml 2026-02-19T20:32:18.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:18.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=160 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 160 = 160 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_38 snap 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:32:18.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:19.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_38@snap 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_38': 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11dfc5a0212b 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11dfc5a0212b 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:32:19.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:05 2026 2026-02-19T20:32:19.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:05 2026 2026-02-19T20:32:19.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:05 2026 2026-02-19T20:32:19.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:32:19.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:32:19.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:32:19.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: bfe65774-7a8a-43f3-a6a6-84525bb3df1b 2026-02-19T20:32:19.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:32:19.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:19.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_38 snap 2026-02-19T20:32:19.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:19.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:32:19.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:32:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_38.export 2026-02-19T20:32:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.export 2026-02-19T20:32:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_38.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.export 2026-02-19T20:32:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_38@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_38.export 2026-02-19T20:32:20.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:20.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_38@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.export 2026-02-19T20:32:20.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:20.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_38.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.export 2026-02-19T20:32:20.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_38.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_38.export 2026-02-19T20:32:20.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:20.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:32:20.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_39 2026-02-19T20:32:20.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_39 snap 2026-02-19T20:32:20.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:20.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:20.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:20.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:20.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_39@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:32:20.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_39 2026-02-19T20:32:20.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_39 started 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:20.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:21.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_39 started 2026-02-19T20:32:21.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:21.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_39 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:32:21.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:32:21.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:32:21.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:21.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:32:21.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:21.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:21.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:21.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_39 2026-02-19T20:32:21.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:21.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:21.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:21.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:21.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:21.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:21.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:21.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:21.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:21.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:21.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:21.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:21.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:21.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:21.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_39 2026-02-19T20:32:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2026-02-19T20:32:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:21.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:21.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_39 2026-02-19T20:32:21.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:32:21.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:21.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:21.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:32:21.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:32:21.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:21.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:32:21.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:32:21.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:32:21.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:21.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:21.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:21.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_39 2026-02-19T20:32:21.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2026-02-19T20:32:21.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:21.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:21.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:21.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_39 2026-02-19T20:32:21.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:32:21.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:21.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:21.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:32:21.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:32:21.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:32:21.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:32:21.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:32:21.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:21.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:21.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:21.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_39 2026-02-19T20:32:21.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_39'] 2026-02-19T20:32:21.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:21.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:21.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:21.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_39 2026-02-19T20:32:21.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:32:21.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:21.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:21.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:21.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:21.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:32:21.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:32:21.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:32:21.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:21.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:21.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:21.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_39 2026-02-19T20:32:21.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:32:21.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:32:21.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:32:21.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:32:21.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:32:21.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:32:21.565 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_39 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_39 2026-02-19T20:32:21.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:21.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:21.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:21.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_39 2026-02-19T20:32:21.740 INFO:tasks.ceph.cluster2.osd.1.trial080.stderr:2026-02-19T20:32:21.738+0000 7f1a949a9640 -1 reset not still connected to 0x4e1d450 2026-02-19T20:32:22.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 162 2026-02-19T20:32:22.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_39 2026-02-19T20:32:22.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:22.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:22.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:22.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:22.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:22.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_39.status 2026-02-19T20:32:22.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_39.status 2026-02-19T20:32:22.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:22.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_39.status 2026-02-19T20:32:22.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_39.status 2026-02-19T20:32:22.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_39.status 2026-02-19T20:32:22.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:22.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.status 2026-02-19T20:32:22.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_39 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_39.status 2026-02-19T20:32:22.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:22.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:22.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:22.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_39.status 2026-02-19T20:32:22.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_39 --format xml 2026-02-19T20:32:22.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:22.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:32:22.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=162 2026-02-19T20:32:22.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:22.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:22.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:32:22.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_39 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.status 2026-02-19T20:32:22.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:22.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:22.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:22.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.status 2026-02-19T20:32:22.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_39 --format xml 2026-02-19T20:32:22.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:22.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:22.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:22.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:22.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:22.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:22.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:22.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 162 = '' 2026-02-19T20:32:22.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:22.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:32:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_39 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.status 2026-02-19T20:32:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.status 2026-02-19T20:32:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_39 --format xml 2026-02-19T20:32:23.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:23.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:23.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:23.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:23.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:23.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:23.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:23.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 162 = '' 2026-02-19T20:32:23.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:23.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:32:23.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_39 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.status 2026-02-19T20:32:23.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:23.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:23.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:23.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.status 2026-02-19T20:32:23.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_39 --format xml 2026-02-19T20:32:23.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:23.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:23.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=162 2026-02-19T20:32:23.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 162 = 162 2026-02-19T20:32:23.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:23.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_39 snap 2026-02-19T20:32:23.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:23.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:23.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:23.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:23.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:23.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:32:23.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:24.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_39@snap 2026-02-19T20:32:24.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_39': 2026-02-19T20:32:24.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:32:24.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:32:24.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:32:24.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123d28742a27 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123d28742a27 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:17 2026 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:17 2026 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:17 2026 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:32:24.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:32:24.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:32:24.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 170be547-d7a9-46a7-b6db-d32c6a98434d 2026-02-19T20:32:24.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:32:24.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:24.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_39 snap 2026-02-19T20:32:24.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:24.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:32:24.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:24.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:32:24.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_39.export 2026-02-19T20:32:24.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.export 2026-02-19T20:32:24.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_39.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.export 2026-02-19T20:32:24.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_39@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_39.export 2026-02-19T20:32:25.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:25.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_39@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.export 2026-02-19T20:32:25.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:25.186 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_39.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.export 2026-02-19T20:32:25.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_39.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_39.export 2026-02-19T20:32:25.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:25.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:32:25.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_40 2026-02-19T20:32:25.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_40 snap 2026-02-19T20:32:25.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:25.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:25.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:25.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:25.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_40@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:32:25.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_40 2026-02-19T20:32:25.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:25.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_40 started 2026-02-19T20:32:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:32:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:25.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:26.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_40 started 2026-02-19T20:32:26.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:26.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:26.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:26.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:32:26.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:32:26.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:32:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_40 2026-02-19T20:32:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:32:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:32:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:32:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:32:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:32:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:26.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:32:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_40 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:26.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:26.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:26.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:26.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:26.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:26.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:26.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_40 2026-02-19T20:32:26.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:32:26.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:32:26.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:32:26.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:32:26.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:32:26.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_40 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:32:26.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:32:26.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_40 2026-02-19T20:32:26.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:26.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:26.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:26.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_40 2026-02-19T20:32:27.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 164 2026-02-19T20:32:27.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_40 2026-02-19T20:32:27.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:27.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:27.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:27.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:27.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:27.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_40.status 2026-02-19T20:32:27.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_40.status 2026-02-19T20:32:27.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:27.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_40.status 2026-02-19T20:32:27.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_40.status 2026-02-19T20:32:27.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_40.status 2026-02-19T20:32:27.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.status 2026-02-19T20:32:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_40 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_40.status 2026-02-19T20:32:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_40.status 2026-02-19T20:32:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_40 --format xml 2026-02-19T20:32:27.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:27.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:32:27.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=164 2026-02-19T20:32:27.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:27.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:27.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:32:27.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_40 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.status 2026-02-19T20:32:27.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:27.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:27.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:27.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.status 2026-02-19T20:32:27.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_40 --format xml 2026-02-19T20:32:27.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:27.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:27.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=74 2026-02-19T20:32:27.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 164 = 74 2026-02-19T20:32:27.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:27.948 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:32:28.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_40 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.status 2026-02-19T20:32:28.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:28.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:28.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:28.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.status 2026-02-19T20:32:28.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_40 --format xml 2026-02-19T20:32:28.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:28.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:28.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=74 2026-02-19T20:32:28.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 164 = 74 2026-02-19T20:32:28.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:28.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:32:29.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_40 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.status 2026-02-19T20:32:29.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:29.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:29.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:29.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.status 2026-02-19T20:32:29.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_40 --format xml 2026-02-19T20:32:29.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:29.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:29.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:29.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:29.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:29.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:29.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:29.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 164 = '' 2026-02-19T20:32:29.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:29.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:32:30.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_40 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.status 2026-02-19T20:32:30.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:30.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:30.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:30.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.status 2026-02-19T20:32:30.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_40 --format xml 2026-02-19T20:32:30.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:30.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:30.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=164 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 164 = 164 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_40 snap 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:32:30.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:31.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_40@snap 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_40': 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11df8327f60a 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11df8327f60a 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:58 2026 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:58 2026 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:58 2026 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 1f232c6e-6e25-4b6b-89c0-3442feb4127a 2026-02-19T20:32:31.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:32:31.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:31.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_40 snap 2026-02-19T20:32:31.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:31.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:32:31.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:31.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:32:31.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_40.export 2026-02-19T20:32:31.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.export 2026-02-19T20:32:31.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_40.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.export 2026-02-19T20:32:31.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_40@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_40.export 2026-02-19T20:32:32.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:32.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_40@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.export 2026-02-19T20:32:32.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:32.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_40.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.export 2026-02-19T20:32:32.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_40.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_40.export 2026-02-19T20:32:32.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:32.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:32:32.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_41 2026-02-19T20:32:32.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_41 snap 2026-02-19T20:32:32.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:32.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:32.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:32.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:32.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_41@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:32:32.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_41 2026-02-19T20:32:32.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:32.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:32.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:32.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_41 started 2026-02-19T20:32:32.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:32.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:32.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:32.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:32:32.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:32.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:32.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:33.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_41 started 2026-02-19T20:32:33.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:33.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:33.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:33.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:32:33.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:32:33.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:32:33.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_41 2026-02-19T20:32:33.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:32:33.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:33.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:32:33.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:32:33.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:32:33.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:33.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:32:33.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:32:33.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:33.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:33.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:33.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_41 2026-02-19T20:32:33.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:33.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:33.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:33.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:33.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:33.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:33.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:33.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:33.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:33.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:33.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:33.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:33.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_41 2026-02-19T20:32:33.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2026-02-19T20:32:33.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:33.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:33.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_41 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:32:33.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:33.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:33.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:33.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:33.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:33.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:33.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:32:33.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:32:33.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:32:33.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:33.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:33.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:33.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_41 2026-02-19T20:32:33.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_41'] 2026-02-19T20:32:33.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:33.455 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:33.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:33.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_41 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:33.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:33.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:33.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:33.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:33.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:33.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:32:33.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:32:33.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:32:33.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:33.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:33.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:33.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_41 2026-02-19T20:32:33.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:32:33.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:32:33.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:32:33.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:32:33.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:32:33.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:32:33.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:32:33.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:33.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_41 2026-02-19T20:32:33.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:33.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:33.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:32:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:32:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_41 2026-02-19T20:32:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_41 2026-02-19T20:32:34.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 166 2026-02-19T20:32:34.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_41 2026-02-19T20:32:34.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:34.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:34.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:34.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:34.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:34.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_41.status 2026-02-19T20:32:34.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_41.status 2026-02-19T20:32:34.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:34.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_41.status 2026-02-19T20:32:34.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_41.status 2026-02-19T20:32:34.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_41.status 2026-02-19T20:32:34.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:34.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.status 2026-02-19T20:32:34.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_41 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_41.status 2026-02-19T20:32:34.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:34.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:34.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:34.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_41.status 2026-02-19T20:32:34.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_41 --format xml 2026-02-19T20:32:34.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:34.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:32:34.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=166 2026-02-19T20:32:34.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:34.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:34.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:32:34.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_41 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.status 2026-02-19T20:32:34.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:34.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:34.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:34.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.status 2026-02-19T20:32:34.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_41 --format xml 2026-02-19T20:32:34.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:34.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:34.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:34.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:34.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:34.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:34.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:34.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 166 = '' 2026-02-19T20:32:34.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:34.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:32:35.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_41 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.status 2026-02-19T20:32:35.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:35.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:35.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:35.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.status 2026-02-19T20:32:35.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_41 --format xml 2026-02-19T20:32:35.021 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:35.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:35.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:35.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:35.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:35.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:35.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:35.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 166 = '' 2026-02-19T20:32:35.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:35.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:32:35.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_41 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.status 2026-02-19T20:32:35.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:35.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:35.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:35.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.status 2026-02-19T20:32:35.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_41 --format xml 2026-02-19T20:32:35.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:35.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:35.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=166 2026-02-19T20:32:35.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 166 = 166 2026-02-19T20:32:35.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:35.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_41 snap 2026-02-19T20:32:35.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:35.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:35.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:35.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:35.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:35.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:32:35.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:36.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_41@snap 2026-02-19T20:32:36.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_41': 2026-02-19T20:32:36.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 125c49d5555c 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.125c49d5555c 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:42 2026 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:42 2026 2026-02-19T20:32:36.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:42 2026 2026-02-19T20:32:36.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:32:36.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:32:36.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:32:36.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: ec6e3c47-0016-414a-92e7-6ab7bcdd379d 2026-02-19T20:32:36.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:32:36.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:36.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_41 snap 2026-02-19T20:32:36.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:36.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:32:36.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:36.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:32:36.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_41.export 2026-02-19T20:32:36.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.export 2026-02-19T20:32:36.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_41.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.export 2026-02-19T20:32:36.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_41@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_41.export 2026-02-19T20:32:37.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:37.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_41@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.export 2026-02-19T20:32:37.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:37.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_41.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.export 2026-02-19T20:32:37.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_41.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_41.export 2026-02-19T20:32:37.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:37.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:32:37.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_42 2026-02-19T20:32:37.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_42 snap 2026-02-19T20:32:37.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:37.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:37.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:37.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:37.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_42@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:32:37.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_42 2026-02-19T20:32:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_42 started 2026-02-19T20:32:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:32:37.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:37.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:37.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:38.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_42 started 2026-02-19T20:32:38.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:38.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_42 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:32:38.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:38.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:32:38.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:38.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_42 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:38.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:38.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:38.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:38.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:38.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:38.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:38.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_42 2026-02-19T20:32:38.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:32:38.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:32:38.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:32:38.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:32:38.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:32:38.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:32:38.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:32:38.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:38.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_42 2026-02-19T20:32:38.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:38.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:38.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:38.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:38.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:38.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:32:38.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:32:38.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_42 2026-02-19T20:32:38.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:38.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:38.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:38.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_42 2026-02-19T20:32:39.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 168 2026-02-19T20:32:39.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_42 2026-02-19T20:32:39.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:39.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:39.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:39.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:39.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:39.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_42.status 2026-02-19T20:32:39.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_42.status 2026-02-19T20:32:39.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:39.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_42.status 2026-02-19T20:32:39.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_42.status 2026-02-19T20:32:39.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_42.status 2026-02-19T20:32:39.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:39.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.status 2026-02-19T20:32:39.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_42 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_42.status 2026-02-19T20:32:39.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:39.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:39.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:39.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_42.status 2026-02-19T20:32:39.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_42 --format xml 2026-02-19T20:32:39.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:39.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:32:39.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=168 2026-02-19T20:32:39.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:39.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:39.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:32:39.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_42 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.status 2026-02-19T20:32:39.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:39.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:39.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:39.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.status 2026-02-19T20:32:39.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_42 --format xml 2026-02-19T20:32:39.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:39.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:39.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=76 2026-02-19T20:32:39.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 168 = 76 2026-02-19T20:32:39.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:39.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:32:40.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_42 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.status 2026-02-19T20:32:40.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:40.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:40.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:40.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.status 2026-02-19T20:32:40.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_42 --format xml 2026-02-19T20:32:40.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:40.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:40.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=76 2026-02-19T20:32:40.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 168 = 76 2026-02-19T20:32:40.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:40.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:32:41.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_42 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.status 2026-02-19T20:32:41.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:41.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:41.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:41.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.status 2026-02-19T20:32:41.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_42 --format xml 2026-02-19T20:32:41.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:41.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:41.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:41.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:41.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:41.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:41.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:41.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 168 = '' 2026-02-19T20:32:41.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:41.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:32:42.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_42 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.status 2026-02-19T20:32:42.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:42.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:42.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:42.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.status 2026-02-19T20:32:42.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_42 --format xml 2026-02-19T20:32:42.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:42.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:42.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=168 2026-02-19T20:32:42.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 168 = 168 2026-02-19T20:32:42.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:42.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_42 snap 2026-02-19T20:32:42.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:42.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:42.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:42.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:42.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:42.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:32:42.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:43.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_42@snap 2026-02-19T20:32:43.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_42': 2026-02-19T20:32:43.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:32:43.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 125cebd521f3 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.125cebd521f3 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:25:54 2026 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:25:54 2026 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:25:54 2026 2026-02-19T20:32:43.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:32:43.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:32:43.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:32:43.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 181745d0-d881-4b58-8036-60e15d40ce25 2026-02-19T20:32:43.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:32:43.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:43.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_42 snap 2026-02-19T20:32:43.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:43.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:32:43.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:43.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:32:43.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_42.export 2026-02-19T20:32:43.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.export 2026-02-19T20:32:43.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_42.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.export 2026-02-19T20:32:43.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_42@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_42.export 2026-02-19T20:32:43.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:44.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_42@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.export 2026-02-19T20:32:44.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:44.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_42.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.export 2026-02-19T20:32:44.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_42.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_42.export 2026-02-19T20:32:44.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:44.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:32:44.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_43 2026-02-19T20:32:44.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_43 snap 2026-02-19T20:32:44.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:44.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:44.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:44.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:44.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_43@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:32:44.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_43 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_43 started 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:44.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_43 started 2026-02-19T20:32:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:32:45.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_43 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:32:45.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:32:45.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_43 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:45.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:45.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:45.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:45.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:45.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:45.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:45.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:45.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:45.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:45.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:45.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:45.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:45.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:45.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:45.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:45.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:45.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_43 2026-02-19T20:32:45.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2026-02-19T20:32:45.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:45.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:45.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:45.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_43 2026-02-19T20:32:45.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:32:45.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:45.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:45.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:45.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:45.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:45.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:45.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:32:45.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:32:45.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:32:45.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:45.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:45.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:45.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_43 2026-02-19T20:32:45.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2026-02-19T20:32:45.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:45.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:45.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:45.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_43 2026-02-19T20:32:45.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:32:45.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:45.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:45.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:45.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:45.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:45.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:45.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:32:45.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:32:45.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:32:45.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:45.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:45.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:45.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_43 2026-02-19T20:32:45.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_43'] 2026-02-19T20:32:45.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:45.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:45.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:45.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_43 2026-02-19T20:32:45.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:32:45.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:45.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:45.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:32:45.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:32:45.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:32:45.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:45.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:45.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:32:45.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:32:45.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:32:45.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:45.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:45.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:32:45.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_43 2026-02-19T20:32:45.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:32:45.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:32:45.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:32:45.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:32:45.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_43 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:45.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:32:45.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:32:45.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_43 2026-02-19T20:32:45.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:45.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:45.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:45.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_43 2026-02-19T20:32:46.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 170 2026-02-19T20:32:46.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_43 2026-02-19T20:32:46.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:46.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:46.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:46.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:46.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:46.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_43.status 2026-02-19T20:32:46.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_43.status 2026-02-19T20:32:46.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:46.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_43.status 2026-02-19T20:32:46.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_43.status 2026-02-19T20:32:46.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_43.status 2026-02-19T20:32:46.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:46.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.status 2026-02-19T20:32:46.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_43 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_43.status 2026-02-19T20:32:46.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:46.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:46.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:46.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_43.status 2026-02-19T20:32:46.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_43 --format xml 2026-02-19T20:32:46.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:46.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:32:46.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=170 2026-02-19T20:32:46.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:46.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:46.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:32:46.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_43 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.status 2026-02-19T20:32:46.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:46.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:46.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:46.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.status 2026-02-19T20:32:46.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_43 --format xml 2026-02-19T20:32:46.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:46.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:46.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=77 2026-02-19T20:32:46.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 170 = 77 2026-02-19T20:32:46.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:46.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:32:47.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_43 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.status 2026-02-19T20:32:47.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:47.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:47.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:47.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.status 2026-02-19T20:32:47.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_43 --format xml 2026-02-19T20:32:47.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:47.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:47.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=77 2026-02-19T20:32:47.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 170 = 77 2026-02-19T20:32:47.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:47.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:32:48.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_43 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.status 2026-02-19T20:32:48.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:48.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:48.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:48.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.status 2026-02-19T20:32:48.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_43 --format xml 2026-02-19T20:32:48.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:48.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:48.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:48.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:48.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:48.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:48.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:48.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 170 = '' 2026-02-19T20:32:48.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:48.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:32:49.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_43 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.status 2026-02-19T20:32:49.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:49.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:49.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:49.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.status 2026-02-19T20:32:49.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_43 --format xml 2026-02-19T20:32:49.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:49.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:49.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=170 2026-02-19T20:32:49.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 170 = 170 2026-02-19T20:32:49.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:49.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_43 snap 2026-02-19T20:32:49.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:49.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:49.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:49.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:49.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:49.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:32:49.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:50.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_43@snap 2026-02-19T20:32:50.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_43': 2026-02-19T20:32:50.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:32:50.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:32:50.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:32:50.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123ddddf27c1 2026-02-19T20:32:50.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123ddddf27c1 2026-02-19T20:32:50.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:32:50.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:32:50.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:32:50.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:32:50.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:08 2026 2026-02-19T20:32:50.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:08 2026 2026-02-19T20:32:50.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:08 2026 2026-02-19T20:32:50.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:32:50.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:32:50.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:32:50.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 786b51f3-eb62-4ad3-80e9-7168fa5f0c2e 2026-02-19T20:32:50.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:32:50.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:50.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_43 snap 2026-02-19T20:32:50.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:50.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:32:50.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:50.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:32:50.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_43.export 2026-02-19T20:32:50.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.export 2026-02-19T20:32:50.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_43.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.export 2026-02-19T20:32:50.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_43@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_43.export 2026-02-19T20:32:51.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:51.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_43@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.export 2026-02-19T20:32:51.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:51.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_43.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.export 2026-02-19T20:32:51.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_43.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_43.export 2026-02-19T20:32:51.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:51.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:32:51.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_44 2026-02-19T20:32:51.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_44 snap 2026-02-19T20:32:51.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:51.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:51.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:51.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:51.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_44@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:32:51.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_44 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_44 started 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:52.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_44 started 2026-02-19T20:32:52.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:52.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:52.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:52.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:32:52.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:32:52.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:32:52.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_44 2026-02-19T20:32:52.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:32:52.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:52.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:32:52.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:32:52.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:32:52.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:32:52.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:52.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:32:52.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:52.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:52.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:52.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_44 2026-02-19T20:32:52.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:52.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:52.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:52.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:52.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:52.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:52.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:52.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:52.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:52.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:52.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_44 2026-02-19T20:32:52.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2026-02-19T20:32:52.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:52.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:52.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:52.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_44 2026-02-19T20:32:52.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:52.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:52.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:32:52.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:32:52.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:52.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:52.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:32:52.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:32:52.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:32:52.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:52.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:52.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:32:52.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_44 2026-02-19T20:32:52.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_44'] 2026-02-19T20:32:52.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:32:52.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:32:52.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:52.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_44 2026-02-19T20:32:52.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:32:52.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:52.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:32:52.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:32:52.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:52.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:52.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:32:52.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:32:52.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:32:52.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:52.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:52.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:32:52.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_44 2026-02-19T20:32:52.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:32:52.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:32:52.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:32:52.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:32:52.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:32:52.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:32:52.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:32:52.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:52.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_44 2026-02-19T20:32:52.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_44 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:52.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_44 2026-02-19T20:32:53.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 172 2026-02-19T20:32:53.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_44 2026-02-19T20:32:53.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:53.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:53.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:53.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:53.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:53.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_44.status 2026-02-19T20:32:53.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_44.status 2026-02-19T20:32:53.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:53.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_44.status 2026-02-19T20:32:53.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_44.status 2026-02-19T20:32:53.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_44.status 2026-02-19T20:32:53.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:32:53.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.status 2026-02-19T20:32:53.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_44 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_44.status 2026-02-19T20:32:53.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:53.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:53.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:53.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_44.status 2026-02-19T20:32:53.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_44 --format xml 2026-02-19T20:32:53.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:53.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:32:53.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=172 2026-02-19T20:32:53.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:53.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:53.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:32:53.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_44 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.status 2026-02-19T20:32:53.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:53.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:53.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:53.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.status 2026-02-19T20:32:53.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_44 --format xml 2026-02-19T20:32:53.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:53.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:53.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=78 2026-02-19T20:32:53.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 172 = 78 2026-02-19T20:32:53.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:53.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:32:54.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_44 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.status 2026-02-19T20:32:54.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:54.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:54.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:54.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.status 2026-02-19T20:32:54.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_44 --format xml 2026-02-19T20:32:54.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:54.389 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:54.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=78 2026-02-19T20:32:54.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 172 = 78 2026-02-19T20:32:54.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:54.391 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:32:55.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_44 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.status 2026-02-19T20:32:55.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:55.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:55.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:55.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.status 2026-02-19T20:32:55.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_44 --format xml 2026-02-19T20:32:55.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:55.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:32:55.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:55.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:32:55.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:32:55.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:32:55.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:32:55.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 172 = '' 2026-02-19T20:32:55.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:32:55.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:32:56.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_44 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.status 2026-02-19T20:32:56.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:56.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:56.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:56.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.status 2026-02-19T20:32:56.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_44 --format xml 2026-02-19T20:32:56.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:32:56.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:32:56.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=172 2026-02-19T20:32:56.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 172 = 172 2026-02-19T20:32:56.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:56.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_44 snap 2026-02-19T20:32:56.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:56.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:56.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:56.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:56.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:56.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:32:56.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:57.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_44@snap 2026-02-19T20:32:57.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_44': 2026-02-19T20:32:57.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:32:57.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:32:57.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:32:57.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 125ccab37886 2026-02-19T20:32:57.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.125ccab37886 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:15 2026 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:15 2026 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:15 2026 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 4b187764-f4c1-4c2e-832e-6fc3fbe30766 2026-02-19T20:32:57.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:32:57.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:57.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_44 snap 2026-02-19T20:32:57.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:57.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:32:57.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:57.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:32:57.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_44.export 2026-02-19T20:32:57.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.export 2026-02-19T20:32:57.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_44.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.export 2026-02-19T20:32:57.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_44@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_44.export 2026-02-19T20:32:58.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:58.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_44@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.export 2026-02-19T20:32:58.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:32:58.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_44.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.export 2026-02-19T20:32:58.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_44.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_44.export 2026-02-19T20:32:58.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:58.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:32:58.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_45 2026-02-19T20:32:58.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_45 snap 2026-02-19T20:32:58.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:58.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:58.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:32:58.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:32:58.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_45@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:32:58.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_45 2026-02-19T20:32:58.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:58.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:58.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:32:58.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_45 started 2026-02-19T20:32:58.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:58.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:58.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:32:58.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:32:58.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:32:58.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:58.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:32:59.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_45 started 2026-02-19T20:32:59.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:32:59.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:59.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_45 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:32:59.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:32:59.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:32:59.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:32:59.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_45 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:32:59.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:32:59.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:32:59.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:32:59.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:59.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:59.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:32:59.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_45 2026-02-19T20:32:59.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:32:59.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:32:59.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:32:59.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:32:59.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:32:59.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:32:59.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:32:59.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:32:59.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_45 2026-02-19T20:32:59.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:32:59.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:59.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:32:59.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:32:59.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:32:59.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:32:59.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:32:59.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_45 2026-02-19T20:32:59.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:32:59.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:32:59.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:32:59.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_45 2026-02-19T20:33:00.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 174 2026-02-19T20:33:00.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_45 2026-02-19T20:33:00.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:00.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:00.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:00.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:00.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:33:00.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_45.status 2026-02-19T20:33:00.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_45.status 2026-02-19T20:33:00.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:00.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_45.status 2026-02-19T20:33:00.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_45.status 2026-02-19T20:33:00.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_45.status 2026-02-19T20:33:00.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.status 2026-02-19T20:33:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_45 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_45.status 2026-02-19T20:33:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:33:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_45.status 2026-02-19T20:33:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_45 --format xml 2026-02-19T20:33:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:00.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:33:00.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=174 2026-02-19T20:33:00.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:00.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:00.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:33:00.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_45 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.status 2026-02-19T20:33:00.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:00.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:00.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:33:00.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.status 2026-02-19T20:33:00.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_45 --format xml 2026-02-19T20:33:00.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:00.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:00.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=79 2026-02-19T20:33:00.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 174 = 79 2026-02-19T20:33:00.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:00.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:33:01.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_45 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.status 2026-02-19T20:33:01.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:01.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:01.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:33:01.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.status 2026-02-19T20:33:01.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_45 --format xml 2026-02-19T20:33:01.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:01.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:01.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=79 2026-02-19T20:33:01.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 174 = 79 2026-02-19T20:33:01.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:01.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:33:02.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_45 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.status 2026-02-19T20:33:02.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:02.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:02.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:33:02.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.status 2026-02-19T20:33:02.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_45 --format xml 2026-02-19T20:33:02.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:02.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:02.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:02.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:33:02.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:33:02.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:33:02.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:33:02.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 174 = '' 2026-02-19T20:33:02.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:02.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:33:03.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_45 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.status 2026-02-19T20:33:03.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:03.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:03.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:33:03.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.status 2026-02-19T20:33:03.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_45 --format xml 2026-02-19T20:33:03.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:03.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:03.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=174 2026-02-19T20:33:03.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 174 = 174 2026-02-19T20:33:03.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:03.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_45 snap 2026-02-19T20:33:03.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:03.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:03.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:33:03.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:03.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:03.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:33:03.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:04.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_45@snap 2026-02-19T20:33:04.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_45': 2026-02-19T20:33:04.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:33:04.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:33:04.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:33:04.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113a7354f40c 2026-02-19T20:33:04.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113a7354f40c 2026-02-19T20:33:04.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:33:04.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:29 2026 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:29 2026 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:29 2026 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 11312853-10d6-49e2-a14a-c1c0aeb67353 2026-02-19T20:33:04.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:33:04.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:04.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_45 snap 2026-02-19T20:33:04.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:04.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:33:04.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:04.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:33:04.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_45.export 2026-02-19T20:33:04.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.export 2026-02-19T20:33:04.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_45.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.export 2026-02-19T20:33:04.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_45@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_45.export 2026-02-19T20:33:05.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:05.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_45@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.export 2026-02-19T20:33:05.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:05.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_45.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.export 2026-02-19T20:33:05.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_45.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_45.export 2026-02-19T20:33:05.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:05.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:05.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_46 2026-02-19T20:33:05.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_46 snap 2026-02-19T20:33:05.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:05.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:05.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:05.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:05.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_46@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:33:05.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_46 2026-02-19T20:33:05.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_46 started 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:05.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:06.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_46 started 2026-02-19T20:33:06.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:06.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_46 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:33:06.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:33:06.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:06.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:33:06.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:33:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:33:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:06.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_46 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:06.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:06.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:06.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:06.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:06.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:06.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:06.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:33:06.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:33:06.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:33:06.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:06.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:06.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:06.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_46 2026-02-19T20:33:06.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_46'] 2026-02-19T20:33:06.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:33:06.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:33:06.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:06.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_46 2026-02-19T20:33:06.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:33:06.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:06.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:33:06.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:33:06.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:06.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:06.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:33:06.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:33:06.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:33:06.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:06.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:06.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:06.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_46 2026-02-19T20:33:06.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:33:06.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:33:06.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:33:06.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:33:06.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:33:06.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:33:06.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:33:06.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:06.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_46 2026-02-19T20:33:06.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:06.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:06.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:06.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:06.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:06.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:33:06.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:33:06.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_46 2026-02-19T20:33:06.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:06.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:06.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:06.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_46 2026-02-19T20:33:07.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 176 2026-02-19T20:33:07.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_46 2026-02-19T20:33:07.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:07.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:07.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:07.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:07.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:07.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_46.status 2026-02-19T20:33:07.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_46.status 2026-02-19T20:33:07.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:07.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_46.status 2026-02-19T20:33:07.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_46.status 2026-02-19T20:33:07.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_46.status 2026-02-19T20:33:07.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:07.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.status 2026-02-19T20:33:07.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_46 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_46.status 2026-02-19T20:33:07.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:07.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:07.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:07.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_46.status 2026-02-19T20:33:07.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_46 --format xml 2026-02-19T20:33:07.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:07.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:33:07.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=176 2026-02-19T20:33:07.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:07.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:07.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:33:07.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_46 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.status 2026-02-19T20:33:07.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:07.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:07.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:07.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.status 2026-02-19T20:33:07.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_46 --format xml 2026-02-19T20:33:07.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:07.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:07.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=80 2026-02-19T20:33:07.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 176 = 80 2026-02-19T20:33:07.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:07.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:33:08.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_46 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.status 2026-02-19T20:33:08.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:08.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:08.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:08.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.status 2026-02-19T20:33:08.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_46 --format xml 2026-02-19T20:33:08.288 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:08.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:08.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=80 2026-02-19T20:33:08.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 176 = 80 2026-02-19T20:33:08.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:08.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:33:09.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_46 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.status 2026-02-19T20:33:09.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:09.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:09.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:09.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.status 2026-02-19T20:33:09.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_46 --format xml 2026-02-19T20:33:09.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:09.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:09.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:09.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:33:09.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:33:09.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:33:09.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:33:09.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 176 = '' 2026-02-19T20:33:09.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:09.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:33:09.729 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:33:09.726+0000 7f635ae8a640 -1 reset not still connected to 0xab68410 2026-02-19T20:33:10.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_46 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.status 2026-02-19T20:33:10.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:10.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:10.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:10.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.status 2026-02-19T20:33:10.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_46 --format xml 2026-02-19T20:33:10.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:10.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:10.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=176 2026-02-19T20:33:10.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 176 = 176 2026-02-19T20:33:10.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:10.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_46 snap 2026-02-19T20:33:10.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:10.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:10.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:10.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:10.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:10.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:33:10.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:11.150 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9993296703023644) > weight (0.25) 2026-02-19T20:33:11.151 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.55110875416803) > weight (0.25) 2026-02-19T20:33:11.151 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2026-02-19T20:33:11.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2026-02-19T20:33:11.151 INFO:tasks.rbd_mirror_thrash:waiting for 2.642138355055245 secs before reviving daemons 2026-02-19T20:33:11.152 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:33:11.150+0000 7f9e0e822640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 48065) UID: 1000 2026-02-19T20:33:11.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_46@snap 2026-02-19T20:33:11.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_46': 2026-02-19T20:33:11.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:33:11.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:33:11.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:33:11.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11df47d979e6 2026-02-19T20:33:11.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11df47d979e6 2026-02-19T20:33:11.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:33:11.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:33:11.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:33:11.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:33:11.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:43 2026 2026-02-19T20:33:11.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:43 2026 2026-02-19T20:33:11.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:43 2026 2026-02-19T20:33:11.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:33:11.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:33:11.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:33:11.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: fa7f549b-1b88-4438-8fba-afa9db93d973 2026-02-19T20:33:11.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:33:11.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:11.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_46 snap 2026-02-19T20:33:11.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:11.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:33:11.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:11.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:33:11.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_46.export 2026-02-19T20:33:11.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.export 2026-02-19T20:33:11.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_46.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.export 2026-02-19T20:33:11.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_46@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_46.export 2026-02-19T20:33:11.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:11.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_46@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.export 2026-02-19T20:33:12.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:12.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_46.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.export 2026-02-19T20:33:12.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_46.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_46.export 2026-02-19T20:33:12.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:12.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:12.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_47 2026-02-19T20:33:12.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_47 snap 2026-02-19T20:33:12.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:12.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:12.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:12.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:12.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_47@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:33:12.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_47 2026-02-19T20:33:12.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:12.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:12.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:12.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_47 started 2026-02-19T20:33:12.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:12.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:12.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:12.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:33:12.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:12.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:12.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_47 started 2026-02-19T20:33:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:13.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_47 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:33:13.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:33:13.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:33:13.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:13.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:33:13.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_47 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:13.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:33:13.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:13.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:13.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:13.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:13.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:13.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:13.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:13.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:33:13.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:33:13.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:33:13.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:13.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:13.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:13.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_47 2026-02-19T20:33:13.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:33:13.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:33:13.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:33:13.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:33:13.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_47 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_47 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:13.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_47 2026-02-19T20:33:13.794 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2026-02-19T20:33:13.794 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:33:13.794 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2026-02-19T20:33:13.794 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2026-02-19T20:33:13.794 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2026-02-19T20:33:13.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2026-02-19T20:33:13.797 INFO:tasks.rbd_mirror_thrash:waiting for 101.39173477484532 secs before thrashing 2026-02-19T20:33:14.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 178 2026-02-19T20:33:14.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_47 2026-02-19T20:33:14.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:14.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:14.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:14.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:14.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:14.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_47.status 2026-02-19T20:33:14.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_47.status 2026-02-19T20:33:14.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:14.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_47.status 2026-02-19T20:33:14.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_47.status 2026-02-19T20:33:14.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_47.status 2026-02-19T20:33:14.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:14.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.status 2026-02-19T20:33:14.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_47 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_47.status 2026-02-19T20:33:14.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:14.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:14.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:14.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_47.status 2026-02-19T20:33:14.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_47 --format xml 2026-02-19T20:33:14.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:14.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:33:14.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=178 2026-02-19T20:33:14.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:14.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:14.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:33:14.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_47 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.status 2026-02-19T20:33:14.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:14.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:14.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:14.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.status 2026-02-19T20:33:14.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_47 --format xml 2026-02-19T20:33:14.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:14.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:14.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=81 2026-02-19T20:33:14.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 178 = 81 2026-02-19T20:33:14.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:14.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:33:15.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_47 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.status 2026-02-19T20:33:15.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:15.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:15.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:15.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.status 2026-02-19T20:33:15.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_47 --format xml 2026-02-19T20:33:15.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:15.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:15.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=81 2026-02-19T20:33:15.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 178 = 81 2026-02-19T20:33:15.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:15.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:33:16.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_47 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.status 2026-02-19T20:33:16.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:16.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:16.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:16.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.status 2026-02-19T20:33:16.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_47 --format xml 2026-02-19T20:33:16.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:16.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:16.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:16.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:33:16.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:33:16.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:33:16.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:33:16.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 178 = '' 2026-02-19T20:33:16.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:16.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:33:17.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_47 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.status 2026-02-19T20:33:17.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:17.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:17.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:17.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.status 2026-02-19T20:33:17.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_47 --format xml 2026-02-19T20:33:17.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:17.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=178 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 178 = 178 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_47 snap 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:33:17.794 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:18.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_47@snap 2026-02-19T20:33:18.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_47': 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 123d2fc89864 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.123d2fc89864 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:26:55 2026 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:26:55 2026 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:26:55 2026 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 9bf92945-bae6-44d1-9712-24406b283d4b 2026-02-19T20:33:18.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:33:18.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:18.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_47 snap 2026-02-19T20:33:18.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:18.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:33:18.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:18.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:33:18.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_47.export 2026-02-19T20:33:18.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.export 2026-02-19T20:33:18.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_47.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.export 2026-02-19T20:33:18.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_47@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_47.export 2026-02-19T20:33:18.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:18.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_47@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.export 2026-02-19T20:33:19.083 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_47.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.export 2026-02-19T20:33:19.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_47.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_47.export 2026-02-19T20:33:19.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:19.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:19.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_48 2026-02-19T20:33:19.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_48 snap 2026-02-19T20:33:19.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:19.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:19.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:19.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:19.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_48@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:33:19.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_48 2026-02-19T20:33:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_48 started 2026-02-19T20:33:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:33:19.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:19.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:19.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:20.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_48 started 2026-02-19T20:33:20.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:20.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:20.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:20.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:33:20.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:33:20.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:33:20.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_48 2026-02-19T20:33:20.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:33:20.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:33:20.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:20.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:33:20.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:33:20.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:33:20.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:20.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:33:20.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:33:20.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:33:20.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:20.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_48 2026-02-19T20:33:20.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:33:20.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:33:20.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:20.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:20.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:33:20.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:33:20.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:33:20.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:20.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:20.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:20.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_48 2026-02-19T20:33:20.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_48'] 2026-02-19T20:33:20.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:33:20.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:33:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_48 2026-02-19T20:33:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:33:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:33:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:33:20.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:20.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:20.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:33:20.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:33:20.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:33:20.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:20.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:20.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:20.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_48 2026-02-19T20:33:20.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_48'] 2026-02-19T20:33:20.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:33:20.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_48 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:20.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:33:20.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:33:20.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:33:20.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:33:20.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:33:20.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:33:20.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_48 2026-02-19T20:33:20.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:33:20.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:33:20.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:33:20.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:33:20.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:33:20.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:33:20.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:33:20.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:20.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_48 2026-02-19T20:33:20.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:20.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:20.809 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:20.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:20.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:20.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:33:20.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:33:20.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_48 2026-02-19T20:33:20.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:20.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:20.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:20.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_48 2026-02-19T20:33:21.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 180 2026-02-19T20:33:21.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_48 2026-02-19T20:33:21.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:21.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:21.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:21.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:21.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:21.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_48.status 2026-02-19T20:33:21.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_48.status 2026-02-19T20:33:21.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:21.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_48.status 2026-02-19T20:33:21.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_48.status 2026-02-19T20:33:21.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_48.status 2026-02-19T20:33:21.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:21.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.status 2026-02-19T20:33:21.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_48 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_48.status 2026-02-19T20:33:21.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:21.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:21.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:21.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_48.status 2026-02-19T20:33:21.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_48 --format xml 2026-02-19T20:33:21.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:21.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:33:21.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=180 2026-02-19T20:33:21.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:21.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:21.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:33:21.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_48 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.status 2026-02-19T20:33:21.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:21.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:21.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:21.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.status 2026-02-19T20:33:21.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_48 --format xml 2026-02-19T20:33:21.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:21.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:21.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=82 2026-02-19T20:33:21.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 180 = 82 2026-02-19T20:33:21.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:21.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:33:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_48 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.status 2026-02-19T20:33:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.status 2026-02-19T20:33:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_48 --format xml 2026-02-19T20:33:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:22.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:22.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=82 2026-02-19T20:33:22.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 180 = 82 2026-02-19T20:33:22.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:22.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:33:23.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_48 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.status 2026-02-19T20:33:23.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:23.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:23.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:23.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.status 2026-02-19T20:33:23.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_48 --format xml 2026-02-19T20:33:23.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:23.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:23.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:23.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:33:23.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:33:23.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:33:23.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:33:23.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 180 = '' 2026-02-19T20:33:23.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:23.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:33:24.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_48 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.status 2026-02-19T20:33:24.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:24.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:24.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:24.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.status 2026-02-19T20:33:24.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_48 --format xml 2026-02-19T20:33:24.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:24.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:24.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=180 2026-02-19T20:33:24.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 180 = 180 2026-02-19T20:33:24.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:24.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_48 snap 2026-02-19T20:33:24.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:24.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:24.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:24.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:24.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:24.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:33:24.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:25.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_48@snap 2026-02-19T20:33:25.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_48': 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 125ccfff86db 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.125ccfff86db 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:27:05 2026 2026-02-19T20:33:25.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:27:05 2026 2026-02-19T20:33:25.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:27:05 2026 2026-02-19T20:33:25.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:33:25.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:33:25.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:33:25.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 893ecffe-1a73-4473-a8e4-b92236e68f8f 2026-02-19T20:33:25.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:33:25.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:25.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_48 snap 2026-02-19T20:33:25.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:25.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:33:25.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:25.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:33:25.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_48.export 2026-02-19T20:33:25.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.export 2026-02-19T20:33:25.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_48.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.export 2026-02-19T20:33:25.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_48@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_48.export 2026-02-19T20:33:25.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:25.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_48@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.export 2026-02-19T20:33:26.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:26.126 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_48.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.export 2026-02-19T20:33:26.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_48.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_48.export 2026-02-19T20:33:26.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:26.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:26.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_49 2026-02-19T20:33:26.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_49 snap 2026-02-19T20:33:26.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:26.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:26.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:26.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:26.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_49@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:33:26.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_49 2026-02-19T20:33:26.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_49 started 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:26.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:27.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_49 started 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_49 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:27.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:33:27.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:33:27.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:33:27.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:33:27.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:27.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:33:27.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:33:27.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:33:27.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:27.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_49 2026-02-19T20:33:27.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:33:27.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:27.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:27.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:33:27.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:33:27.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:33:27.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:33:27.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:33:27.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:27.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:27.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:27.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_49 2026-02-19T20:33:27.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_49'] 2026-02-19T20:33:27.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:33:27.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:33:27.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:27.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_49 2026-02-19T20:33:27.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:33:27.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:27.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:33:27.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:33:27.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:27.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:27.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:33:27.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:33:27.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:33:27.366 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:27.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:27.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:27.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_49 2026-02-19T20:33:27.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_49'] 2026-02-19T20:33:27.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:33:27.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:33:27.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:27.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:2 rbd mirror status mirror/image_49 2026-02-19T20:33:27.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:2 2026-02-19T20:33:27.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:27.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:27.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:2 cluster instance 2026-02-19T20:33:27.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:2 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=2 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 2 = cluster1:2 ']' 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=2 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:2 cluster1 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:2 2026-02-19T20:33:27.414 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:2 local_cluster instance 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:2 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=2 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 2 = cluster1:2 ']' 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:27.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=2 2026-02-19T20:33:27.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=2 2026-02-19T20:33:27.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.2 'admin socket' 2026-02-19T20:33:27.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:33:27.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:33:27.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2026-02-19T20:33:27.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/image_49 2026-02-19T20:33:27.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_49'] 2026-02-19T20:33:27.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:33:27.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:33:27.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:27.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:3 rbd mirror status mirror/image_49 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:3 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:3 cluster instance 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:3 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=3 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 3 = cluster1:3 ']' 2026-02-19T20:33:27.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=3 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:3 cluster1 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:3 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:3 local_cluster instance 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:3 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:27.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:27.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=3 2026-02-19T20:33:27.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 3 = cluster1:3 ']' 2026-02-19T20:33:27.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:27.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:27.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=3 2026-02-19T20:33:27.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=3 2026-02-19T20:33:27.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.3 'admin socket' 2026-02-19T20:33:27.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:33:27.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:33:27.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2026-02-19T20:33:27.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/image_49 2026-02-19T20:33:27.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:33:27.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:33:27.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:33:27.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:33:27.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:33:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:33:27.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_49 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_49 2026-02-19T20:33:27.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:27.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:27.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:27.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_49 2026-02-19T20:33:28.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9968253310864486) > weight (0.25) 2026-02-19T20:33:28.122 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4883871836702005) > weight (0.25) 2026-02-19T20:33:28.122 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9236267126775762) > weight (0.25) 2026-02-19T20:33:28.122 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8762914781248967) > weight (0.25) 2026-02-19T20:33:28.122 INFO:tasks.rbd_mirror_thrash:waiting for 88.43193315029521 secs before thrashing 2026-02-19T20:33:28.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 182 2026-02-19T20:33:28.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_49 2026-02-19T20:33:28.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:28.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:28.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:28.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:28.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:28.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_49.status 2026-02-19T20:33:28.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_49.status 2026-02-19T20:33:28.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:28.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_49.status 2026-02-19T20:33:28.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_49.status 2026-02-19T20:33:28.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_49.status 2026-02-19T20:33:28.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:28.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.status 2026-02-19T20:33:28.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_49 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_49.status 2026-02-19T20:33:28.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:28.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:28.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:28.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_49.status 2026-02-19T20:33:28.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_49 --format xml 2026-02-19T20:33:28.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:28.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:33:28.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=182 2026-02-19T20:33:28.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:28.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:28.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:33:28.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_49 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.status 2026-02-19T20:33:28.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:28.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:28.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:28.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.status 2026-02-19T20:33:28.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_49 --format xml 2026-02-19T20:33:28.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:28.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:28.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=83 2026-02-19T20:33:28.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 182 = 83 2026-02-19T20:33:28.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:28.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:33:28.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_49 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.status 2026-02-19T20:33:28.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:28.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:28.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:28.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.status 2026-02-19T20:33:28.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_49 --format xml 2026-02-19T20:33:28.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:29.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:29.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:29.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:33:29.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:33:29.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:33:29.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:33:29.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 182 = '' 2026-02-19T20:33:29.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:29.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:33:29.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_49 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.status 2026-02-19T20:33:29.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:29.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:29.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:29.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.status 2026-02-19T20:33:29.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_49 --format xml 2026-02-19T20:33:29.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:29.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=182 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 182 = 182 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_49 snap 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:29.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:33:29.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:30.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_49@snap 2026-02-19T20:33:30.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_49': 2026-02-19T20:33:30.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:33:30.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:33:30.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:33:30.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 113a4b9a80d5 2026-02-19T20:33:30.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.113a4b9a80d5 2026-02-19T20:33:30.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:33:30.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:33:30.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:33:30.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:33:30.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:27:18 2026 2026-02-19T20:33:30.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:27:18 2026 2026-02-19T20:33:30.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:27:18 2026 2026-02-19T20:33:30.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:33:30.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:33:30.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:33:30.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 6b13aa42-62d2-4bf0-b934-8f6d6bb3c8da 2026-02-19T20:33:30.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:33:30.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:30.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_49 snap 2026-02-19T20:33:30.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:30.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:33:30.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:30.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:33:30.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_49.export 2026-02-19T20:33:30.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.export 2026-02-19T20:33:30.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_49.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.export 2026-02-19T20:33:30.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_49@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_49.export 2026-02-19T20:33:30.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:31.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_49@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.export 2026-02-19T20:33:31.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:31.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_49.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.export 2026-02-19T20:33:31.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_49.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_49.export 2026-02-19T20:33:31.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:31.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:31.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_50 2026-02-19T20:33:31.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_snap cluster2 mirror image_50 snap 2026-02-19T20:33:31.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:31.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:31.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:31.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:31.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror snap create image_50@snap --debug-rbd=20 --debug-journaler=20 2026-02-19T20:33:31.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_started cluster1 mirror image_50 2026-02-19T20:33:31.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:31.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:31.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:31.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_replay_state cluster1 mirror image_50 started 2026-02-19T20:33:31.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:31.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:31.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:31.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=started 2026-02-19T20:33:31.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:31.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:31.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:32.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_replay_state cluster1 mirror image_50 started 2026-02-19T20:33:32.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:32.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:32.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:32.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=started 2026-02-19T20:33:32.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_result 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=stopped 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemons cluster1 rbd mirror status mirror/image_50 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_instance=cluster1 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance=cluster1 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local loop_instance 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for s in 0 1 2 4 8 8 8 8 8 8 8 8 16 16 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sleep 0 2026-02-19T20:33:32.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -i state 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' cluster1 '!=' cluster1 ']' 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ seq 0 3 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:0 rbd mirror status mirror/image_50 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:0 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:0 cluster instance 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:0 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:32.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=0 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:0 cluster1 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:0 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:32.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:0 local_cluster instance 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:0 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=0 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 0 = cluster1:0 ']' 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=0 2026-02-19T20:33:32.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=0 2026-02-19T20:33:32.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.0 'admin socket' 2026-02-19T20:33:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2026-02-19T20:33:32.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/image_50 2026-02-19T20:33:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:Invalid command: unused arguments: ['mirror/image_50'] 2026-02-19T20:33:32.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd mirror status : get status for rbd mirror 2026-02-19T20:33:32.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:admin_socket: invalid command 2026-02-19T20:33:32.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ for loop_instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ admin_daemon cluster1:1 rbd mirror status mirror/image_50 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1:1 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ shift 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ set_cluster_instance cluster1:1 cluster instance 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local val=cluster1:1 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster_var_name=cluster 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local instance_var_name=instance 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:33:32.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=1 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ '[' 1 = cluster1:1 ']' 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval cluster=cluster1 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ eval instance=1 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ daemon_asok_file cluster1:1 cluster1 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local local_cluster=cluster1:1 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster=cluster1 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ set_cluster_instance cluster1:1 local_cluster instance 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local val=cluster1:1 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local cluster_var_name=local_cluster 2026-02-19T20:33:32.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ local instance_var_name=instance 2026-02-19T20:33:32.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ cluster=cluster1 2026-02-19T20:33:32.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ instance=1 2026-02-19T20:33:32.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ '[' 1 = cluster1:1 ']' 2026-02-19T20:33:32.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval local_cluster=cluster1 2026-02-19T20:33:32.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ local_cluster=cluster1 2026-02-19T20:33:32.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ eval instance=1 2026-02-19T20:33:32.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ instance=1 2026-02-19T20:33:32.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++++ ceph-conf --cluster cluster1 --name client.mirror.1 'admin socket' 2026-02-19T20:33:32.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+++ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:32.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:32.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2026-02-19T20:33:32.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/image_50 2026-02-19T20:33:32.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ return 0 2026-02-19T20:33:32.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ status_result=' "state": "Replaying"' 2026-02-19T20:33:32.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo ' "state": "Replaying"' 2026-02-19T20:33:32.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ grep -i Replaying 2026-02-19T20:33:32.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: "state": "Replaying" 2026-02-19T20:33:32.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=started 2026-02-19T20:33:32.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test started = started 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror mirror image_50 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = journal ']' 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' snapshot = snapshot ']' 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ mirror_image_snapshot cluster2 mirror image_50 2026-02-19T20:33:32.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:32.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:32.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:32.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/image_50 2026-02-19T20:33:33.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Snapshot ID: 184 2026-02-19T20:33:33.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror mirror image_50 2026-02-19T20:33:33.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_cluster=cluster1 2026-02-19T20:33:33.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:33.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_pool=mirror 2026-02-19T20:33:33.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local remote_pool=mirror 2026-02-19T20:33:33.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:33.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster2-mirror-image_50.status 2026-02-19T20:33:33.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster2-mirror-image_50.status 2026-02-19T20:33:33.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:33.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local status_log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_50.status 2026-02-19T20:33:33.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ mkfname cluster1-mirror-image_50.status 2026-02-19T20:33:33.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ echo cluster1-mirror-image_50.status 2026-02-19T20:33:33.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -e 's|[/ ]|_|g' 2026-02-19T20:33:33.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local local_status_log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.status 2026-02-19T20:33:33.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster2 mirror image_50 /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_50.status 2026-02-19T20:33:33.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:33.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:33.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:33.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_50.status 2026-02-19T20:33:33.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap list --all mirror/image_50 --format xml 2026-02-19T20:33:33.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:33.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/id 2026-02-19T20:33:33.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snapshot_id=184 2026-02-19T20:33:33.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:33.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:33.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.2 2026-02-19T20:33:33.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_50 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.status 2026-02-19T20:33:33.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:33.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:33.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:33.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.status 2026-02-19T20:33:33.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_50 --format xml 2026-02-19T20:33:33.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:33.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:33.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=84 2026-02-19T20:33:33.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 184 = 84 2026-02-19T20:33:33.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:33.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.4 2026-02-19T20:33:34.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_50 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.status 2026-02-19T20:33:34.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:34.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:34.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:34.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.status 2026-02-19T20:33:34.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_50 --format xml 2026-02-19T20:33:34.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:34.381 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:34.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=84 2026-02-19T20:33:34.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 184 = 84 2026-02-19T20:33:34.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:34.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.8 2026-02-19T20:33:35.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_50 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.status 2026-02-19T20:33:35.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:35.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:35.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:35.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.status 2026-02-19T20:33:35.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_50 --format xml 2026-02-19T20:33:35.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:35.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:33:35.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:35.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:-:1.1: Document is empty 2026-02-19T20:33:35.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: 2026-02-19T20:33:35.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:^ 2026-02-19T20:33:35.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id= 2026-02-19T20:33:35.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 184 = '' 2026-02-19T20:33:35.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.2 0.4 0.8 1.6 2 2 4 4 8 8 16 16 32 32 2026-02-19T20:33:35.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1.6 2026-02-19T20:33:36.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ get_newest_mirror_snapshot cluster1 mirror image_50 /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.status 2026-02-19T20:33:36.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:36.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:36.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:36.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local log=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.status 2026-02-19T20:33:36.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 snap list --all mirror/image_50 --format xml 2026-02-19T20:33:36.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ xmlstarlet sel -t -c '//snapshots/snapshot[namespace/complete='\''true'\'' and position()=last()]' 2026-02-19T20:33:36.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2026-02-19T20:33:36.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local primary_snapshot_id=184 2026-02-19T20:33:36.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 184 = 184 2026-02-19T20:33:36.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:36.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_snap_present cluster1 mirror image_50 snap 2026-02-19T20:33:36.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:33:36.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:36.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:36.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:36.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:33:36.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 1 2 4 8 8 8 8 8 8 8 8 16 16 16 16 32 32 32 32 2026-02-19T20:33:36.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:33:37.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 info mirror/image_50@snap 2026-02-19T20:33:37.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_50': 2026-02-19T20:33:37.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 128 MiB in 32 objects 2026-02-19T20:33:37.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:33:37.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 2 2026-02-19T20:33:37.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 11dffd5a96c2 2026-02-19T20:33:37.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.11dffd5a96c2 2026-02-19T20:33:37.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering, non-primary 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:27:31 2026 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:27:31 2026 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:27:31 2026 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: protected: False 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 5f2c96e0-e5f7-4818-9461-1ddee61be14f 2026-02-19T20:33:37.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: false 2026-02-19T20:33:37.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:37.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ compare_image_snaps mirror image_50 snap 2026-02-19T20:33:37.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:37.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:33:37.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap_name=snap 2026-02-19T20:33:37.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local ret=0 2026-02-19T20:33:37.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local rmt_export=/tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_50.export 2026-02-19T20:33:37.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local loc_export=/tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.export 2026-02-19T20:33:37.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_50.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.export 2026-02-19T20:33:37.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 -p mirror export image_50@snap /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_50.export 2026-02-19T20:33:37.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:38.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster1 -p mirror export image_50@snap /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.export 2026-02-19T20:33:38.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Exporting image: 3% complete... Exporting image: 6% complete... Exporting image: 9% complete... Exporting image: 12% complete... Exporting image: 15% complete... Exporting image: 18% complete... Exporting image: 21% complete... Exporting image: 25% complete... Exporting image: 28% complete... Exporting image: 31% complete... Exporting image: 34% complete... Exporting image: 37% complete... Exporting image: 40% complete... Exporting image: 43% complete... Exporting image: 46% complete... Exporting image: 50% complete... Exporting image: 53% complete... Exporting image: 56% complete... Exporting image: 59% complete... Exporting image: 62% complete... Exporting image: 65% complete... Exporting image: 68% complete... Exporting image: 71% complete... Exporting image: 75% complete... Exporting image: 78% complete... Exporting image: 81% complete... Exporting image: 84% complete... Exporting image: 87% complete... Exporting image: 90% complete... Exporting image: 93% complete... Exporting image: 96% complete... Exporting image: 100% complete...done. 2026-02-19T20:33:38.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cmp /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_50.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.export 2026-02-19T20:33:38.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -f /tmp/tmp.9BL5XQhnTi/cluster2-mirror-image_50.export /tmp/tmp.9BL5XQhnTi/cluster1-mirror-image_50.export 2026-02-19T20:33:38.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:38.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ testlog 'TEST: delete many images' 2026-02-19T20:33:38.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ tee -a /tmp/tmp.9BL5XQhnTi/rbd-mirror.test.log 2026-02-19T20:33:38.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ date '+%F %T' 2026-02-19T20:33:38.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo 2026-02-19 20:33:38 TEST: delete many images 2026-02-19T20:33:38.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:2026-02-19 20:33:38 TEST: delete many images 2026-02-19T20:33:38.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 50 2026-02-19T20:33:38.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:38.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_1 2026-02-19T20:33:38.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_1 snap 2026-02-19T20:33:38.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:38.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:38.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:33:38.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:38.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_1@snap 2026-02-19T20:33:38.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:38.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_1 2026-02-19T20:33:38.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:38.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:38.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:33:38.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:38.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:38.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_1 2026-02-19T20:33:38.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:38.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:38.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:33:38.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_1 2026-02-19T20:33:38.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_1 2026-02-19T20:33:38.427 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:33:38.425+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x38141a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:38.427 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:33:38.425+0000 7f0ab10a6640 -1 rbd::mirror::ImageReplayer: 0x3704280 [3/1bb58447-6e84-49f6-86ad-1db8d6f29daf] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:39.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:39.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:39.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:39.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_2 2026-02-19T20:33:39.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_2 snap 2026-02-19T20:33:39.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:39.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:39.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:33:39.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:39.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_2@snap 2026-02-19T20:33:40.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:40.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_2 2026-02-19T20:33:40.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:40.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:40.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:33:40.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:40.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:40.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_2 2026-02-19T20:33:40.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:40.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:40.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:33:40.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_2 2026-02-19T20:33:40.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_2 2026-02-19T20:33:40.430 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:33:40.428+0000 7f0aaa899640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3814340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:40.430 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:33:40.428+0000 7f0ab10a6640 -1 rbd::mirror::ImageReplayer: 0x3704780 [3/26fa2d67-71a3-4278-b257-25f69a27f0fe] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:41.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:41.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:41.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:41.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_3 2026-02-19T20:33:41.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_3 snap 2026-02-19T20:33:41.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:41.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:41.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:33:41.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:41.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_3@snap 2026-02-19T20:33:42.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:42.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_3 2026-02-19T20:33:42.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:42.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:42.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:33:42.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:42.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:42.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_3 2026-02-19T20:33:42.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:42.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:42.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:33:42.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_3 2026-02-19T20:33:42.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_3 2026-02-19T20:33:42.428 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:33:42.427+0000 7f828ee9a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4003380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:43.400 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:43.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:43.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:43.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_4 2026-02-19T20:33:43.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_4 snap 2026-02-19T20:33:43.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:43.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:43.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:33:43.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:43.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_4@snap 2026-02-19T20:33:44.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:44.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_4 2026-02-19T20:33:44.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:44.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:44.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:33:44.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:44.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:44.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_4 2026-02-19T20:33:44.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:44.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:44.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:33:44.360 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_4 2026-02-19T20:33:44.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_4 2026-02-19T20:33:44.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:33:44.437+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3815520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:44.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:33:44.437+0000 7f0ab18a7640 -1 rbd::mirror::ImageReplayer: 0x1fb8a00 [3/03107192-aafc-4953-a9c6-561e8976e03f] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:45.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:45.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:45.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:45.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_5 2026-02-19T20:33:45.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_5 snap 2026-02-19T20:33:45.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:45.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:45.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:33:45.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:45.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_5@snap 2026-02-19T20:33:46.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:46.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_5 2026-02-19T20:33:46.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:46.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:46.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:33:46.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:46.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:46.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_5 2026-02-19T20:33:46.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:46.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:46.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:33:46.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_5 2026-02-19T20:33:46.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_5 2026-02-19T20:33:46.438 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:46.436+0000 7ff4cd3fc640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a24680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:46.438 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:46.436+0000 7ff4d3408640 -1 rbd::mirror::ImageReplayer: 0x3152000 [3/e1d197e7-fd23-4005-8baf-4a772b01d8ee] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:47.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:47.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:47.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:47.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_6 2026-02-19T20:33:47.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_6 snap 2026-02-19T20:33:47.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:47.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:47.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:33:47.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:47.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_6@snap 2026-02-19T20:33:48.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:48.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_6 2026-02-19T20:33:48.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:48.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:48.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:33:48.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:48.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:48.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_6 2026-02-19T20:33:48.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:48.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:48.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:33:48.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_6 2026-02-19T20:33:48.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_6 2026-02-19T20:33:48.445 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:48.443+0000 7ff4cd3fc640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x33fc1a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:48.445 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:48.443+0000 7ff4d3c09640 -1 rbd::mirror::ImageReplayer: 0x1a61680 [3/3465736d-6dc5-4652-9dad-a7ea19b6ea48] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:48.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:48.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:48.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:48.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_7 2026-02-19T20:33:48.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_7 snap 2026-02-19T20:33:48.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:48.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:48.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:33:48.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:48.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_7@snap 2026-02-19T20:33:49.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:49.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_7 2026-02-19T20:33:49.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:49.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:49.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:33:49.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:49.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:49.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_7 2026-02-19T20:33:49.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:49.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:49.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:33:49.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_7 2026-02-19T20:33:49.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_7 2026-02-19T20:33:49.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:33:49.694+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x38156c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:49.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:33:49.694+0000 7f0ab18a7640 -1 rbd::mirror::ImageReplayer: 0x3704f00 [3/5770e33c-f75b-4af2-9f16-1bf9dbb1a48f] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:50.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:50.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:50.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:50.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_8 2026-02-19T20:33:50.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_8 snap 2026-02-19T20:33:50.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:50.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:50.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:33:50.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:50.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_8@snap 2026-02-19T20:33:51.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:51.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_8 2026-02-19T20:33:51.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:51.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:51.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:33:51.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:51.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:51.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_8 2026-02-19T20:33:51.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:51.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:51.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:33:51.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_8 2026-02-19T20:33:51.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_8 2026-02-19T20:33:51.689 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:33:51.686+0000 7f635ae8a640 -1 reset not still connected to 0x7711860 2026-02-19T20:33:51.690 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:33:51.686+0000 7f635ae8a640 -1 reset not still connected to 0x7711c70 2026-02-19T20:33:51.690 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:33:51.686+0000 7f635ae8a640 -1 reset not still connected to 0x7a72b60 2026-02-19T20:33:51.690 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:33:51.686+0000 7f635ae8a640 -1 reset not still connected to 0x7e16340 2026-02-19T20:33:51.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:51.698+0000 7ff4ccbfb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a26d00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:51.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:51.698+0000 7ff4d3c09640 -1 rbd::mirror::ImageReplayer: 0x3a66280 [3/c9480ce4-7cc9-491d-9e71-9527ddc66985] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:52.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:52.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:52.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:52.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_9 2026-02-19T20:33:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_9 snap 2026-02-19T20:33:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:33:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:52.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_9@snap 2026-02-19T20:33:53.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:53.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_9 2026-02-19T20:33:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:33:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:53.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_9 2026-02-19T20:33:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:33:53.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_9 2026-02-19T20:33:53.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_9 2026-02-19T20:33:53.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:53.701+0000 7ff4ccbfb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a269c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:53.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:53.701+0000 7ff4d3c09640 -1 rbd::mirror::ImageReplayer: 0x1a61900 [3/96e815a6-741f-4db1-bf35-a75f0477bd8e] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:54.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:54.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:54.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:54.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_10 2026-02-19T20:33:54.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_10 snap 2026-02-19T20:33:54.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:54.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:54.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:33:54.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:54.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_10@snap 2026-02-19T20:33:55.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:55.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_10 2026-02-19T20:33:55.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:55.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:55.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:33:55.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:55.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:55.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_10 2026-02-19T20:33:55.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:55.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:55.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:33:55.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_10 2026-02-19T20:33:55.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_10 2026-02-19T20:33:55.708 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:55.707+0000 7ff4cd3fc640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a26ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:55.708 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:33:55.707+0000 7ff4d3c09640 -1 rbd::mirror::ImageReplayer: 0x1a61b80 [3/b4b55785-9125-4cba-be08-1b555be092a3] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:56.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:56.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:56.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:56.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_11 2026-02-19T20:33:56.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_11 snap 2026-02-19T20:33:56.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:56.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:56.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:33:56.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:56.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_11@snap 2026-02-19T20:33:57.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:57.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_11 2026-02-19T20:33:57.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:57.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:57.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:33:57.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:57.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:57.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_11 2026-02-19T20:33:57.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:57.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:57.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:33:57.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_11 2026-02-19T20:33:57.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_11 2026-02-19T20:33:57.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:33:57.713+0000 7f828e699640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x39596c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:33:57.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:33:57.713+0000 7f8294ea6640 -1 rbd::mirror::ImageReplayer: 0x38f3400 [3/d20ca347-64b1-43f6-9ef4-b6887cc47d4b] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:33:58.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:33:58.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:33:58.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:33:58.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_12 2026-02-19T20:33:58.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_12 snap 2026-02-19T20:33:58.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:58.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:58.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:33:58.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:33:58.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_12@snap 2026-02-19T20:33:59.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:33:59.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_12 2026-02-19T20:33:59.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:59.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:59.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:33:59.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:33:59.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:33:59.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_12 2026-02-19T20:33:59.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:33:59.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:33:59.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:33:59.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_12 2026-02-19T20:33:59.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_12 2026-02-19T20:33:59.715 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:33:59.713+0000 7f828e699640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4001860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:00.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:00.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:00.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:00.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_13 2026-02-19T20:34:00.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_13 snap 2026-02-19T20:34:00.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:00.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:00.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:34:00.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:00.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_13@snap 2026-02-19T20:34:01.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:01.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_13 2026-02-19T20:34:01.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:01.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:01.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:34:01.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:01.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:01.642 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_13 2026-02-19T20:34:01.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:01.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:01.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:34:01.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_13 2026-02-19T20:34:01.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_13 2026-02-19T20:34:01.720 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:01.718+0000 7f828e699640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x40011e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:01.720 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:01.718+0000 7f8294ea6640 -1 rbd::mirror::ImageReplayer: 0x38f2c80 [3/b748d40b-d69e-4188-bd35-09d03d9a991d] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:02.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:02.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:02.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:02.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_14 2026-02-19T20:34:02.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_14 snap 2026-02-19T20:34:02.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:02.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:02.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:34:02.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:02.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_14@snap 2026-02-19T20:34:03.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_14 2026-02-19T20:34:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:34:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:03.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:03.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_14 2026-02-19T20:34:03.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:03.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:03.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:34:03.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_14 2026-02-19T20:34:03.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_14 2026-02-19T20:34:03.725 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:03.723+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3815860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:03.725 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:03.723+0000 7f0ab18a7640 -1 rbd::mirror::ImageReplayer: 0x3704c80 [3/47d7aec0-eb62-49a0-9f34-a0c1ec276d42] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:04.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:04.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:04.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:04.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_15 2026-02-19T20:34:04.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_15 snap 2026-02-19T20:34:04.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:04.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:04.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:34:04.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:04.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_15@snap 2026-02-19T20:34:05.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:05.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_15 2026-02-19T20:34:05.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:05.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:05.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:34:05.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:05.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:05.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_15 2026-02-19T20:34:05.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:05.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:05.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:34:05.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_15 2026-02-19T20:34:05.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_15 2026-02-19T20:34:05.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:05.728+0000 7f5fd0aeb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4dce4e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:05.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:05.728+0000 7f5fd72f8640 -1 rbd::mirror::ImageReplayer: 0x2b2db80 [3/c0ee1cc7-c443-4e99-9bcc-31e4eda9076d] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:06.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:06.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:06.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:06.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_16 2026-02-19T20:34:06.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_16 snap 2026-02-19T20:34:06.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:06.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:06.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:34:06.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:06.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_16@snap 2026-02-19T20:34:07.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:07.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_16 2026-02-19T20:34:07.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:07.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:07.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:34:07.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:07.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:07.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_16 2026-02-19T20:34:07.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:07.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:07.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:34:07.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_16 2026-02-19T20:34:07.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_16 2026-02-19T20:34:07.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:07.735+0000 7f5fd0aeb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4e5c820 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:07.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:07.735+0000 7f5fd6af7640 -1 rbd::mirror::ImageReplayer: 0x4a22a00 [3/97f2f53e-69ec-439c-87a1-d3ef7d099edc] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:08.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:08.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_17 2026-02-19T20:34:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_17 snap 2026-02-19T20:34:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:34:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:08.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_17@snap 2026-02-19T20:34:09.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:09.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_17 2026-02-19T20:34:09.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:09.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:09.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:34:09.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:09.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:09.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_17 2026-02-19T20:34:09.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:09.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:09.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:34:09.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_17 2026-02-19T20:34:09.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_17 2026-02-19T20:34:09.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:09.728+0000 7f828ee9a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3c684e0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:09.730 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:09.728+0000 7f82956a7640 -1 rbd::mirror::ImageReplayer: 0x3e06280 [3/412c65be-fc7c-4de7-ab85-528c3d9910a2] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:10.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:10.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:10.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:10.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_18 2026-02-19T20:34:10.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_18 snap 2026-02-19T20:34:10.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:10.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:10.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:34:10.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:10.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_18@snap 2026-02-19T20:34:11.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:11.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_18 2026-02-19T20:34:11.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:11.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:11.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:34:11.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:11.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:11.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_18 2026-02-19T20:34:11.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:11.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:11.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:34:11.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_18 2026-02-19T20:34:11.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_18 2026-02-19T20:34:11.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:11.736+0000 7f5fd02ea640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4e5c9c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:11.738 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:11.736+0000 7f5fd6af7640 -1 rbd::mirror::ImageReplayer: 0x4a22500 [3/b08bc81f-704a-41ce-b79c-ab730842d310] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:12.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:12.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:12.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:12.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_19 2026-02-19T20:34:12.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_19 snap 2026-02-19T20:34:12.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:12.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:12.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:34:12.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:12.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_19@snap 2026-02-19T20:34:13.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:13.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_19 2026-02-19T20:34:13.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_19 2026-02-19T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:34:13.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_19 2026-02-19T20:34:13.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_19 2026-02-19T20:34:13.942 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:13.940+0000 7ff4ccbfb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x33fc340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:13.942 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:13.940+0000 7ff4d3408640 -1 rbd::mirror::ImageReplayer: 0x3153b80 [3/2bb7364a-f57b-450f-8771-0e4854fa0d34] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:14.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_20 2026-02-19T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_20 snap 2026-02-19T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:14.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_20@snap 2026-02-19T20:34:15.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:15.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_20 2026-02-19T20:34:15.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:15.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:15.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:34:15.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:15.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:15.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_20 2026-02-19T20:34:15.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:15.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:15.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:34:15.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_20 2026-02-19T20:34:15.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_20 2026-02-19T20:34:15.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:15.745+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3815a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:15.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:15.745+0000 7f0ab18a7640 -1 rbd::mirror::ImageReplayer: 0x1fb9180 [3/06be59c7-13b7-4ca9-8bf1-ac1f92b09a6e] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:16.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:16.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:16.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:16.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_21 2026-02-19T20:34:16.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_21 snap 2026-02-19T20:34:16.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:16.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:16.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:34:16.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:16.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_21@snap 2026-02-19T20:34:17.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:17.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_21 2026-02-19T20:34:17.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:17.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:17.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:34:17.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:17.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:17.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_21 2026-02-19T20:34:17.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:17.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:17.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:34:17.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_21 2026-02-19T20:34:17.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_21 2026-02-19T20:34:17.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:17.753+0000 7f5fd02ea640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4e5cb60 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:17.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:17.753+0000 7f5fd6af7640 -1 rbd::mirror::ImageReplayer: 0x4bd2780 [3/6a7c4609-46c1-4458-b2e2-2fc6d6f26028] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:18.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:18.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:18.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:18.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_22 2026-02-19T20:34:18.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_22 snap 2026-02-19T20:34:18.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:18.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:18.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:34:18.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:18.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_22@snap 2026-02-19T20:34:19.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:19.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_22 2026-02-19T20:34:19.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:19.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:19.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:34:19.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:19.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:19.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_22 2026-02-19T20:34:19.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:19.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:19.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:34:19.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_22 2026-02-19T20:34:19.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_22 2026-02-19T20:34:19.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:19.748+0000 7f5fd02ea640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4e5cd00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:19.750 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:19.748+0000 7f5fd6af7640 -1 rbd::mirror::ImageReplayer: 0x4a22780 [3/f9647a4e-5b33-4f6e-bf71-dcbe6192f2bc] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:20.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_23 2026-02-19T20:34:20.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_23 snap 2026-02-19T20:34:20.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:20.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:20.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:34:20.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:20.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_23@snap 2026-02-19T20:34:21.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:21.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_23 2026-02-19T20:34:21.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:21.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:21.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:34:21.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:21.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:21.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_23 2026-02-19T20:34:21.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:21.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:21.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:34:21.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_23 2026-02-19T20:34:21.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_23 2026-02-19T20:34:21.756 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:21.753+0000 7ff4cd3fc640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x368b520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:21.756 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:21.753+0000 7ff4d3408640 -1 rbd::mirror::ImageReplayer: 0x3153180 [3/8b84cd27-943f-470d-8e23-786f49d70f13] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:22.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_24 2026-02-19T20:34:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_24 snap 2026-02-19T20:34:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:34:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:22.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_24@snap 2026-02-19T20:34:23.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:23.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_24 2026-02-19T20:34:23.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:23.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:23.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:34:23.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:23.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:23.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_24 2026-02-19T20:34:23.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:23.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:23.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:34:23.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_24 2026-02-19T20:34:23.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_24 2026-02-19T20:34:23.765 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:23.763+0000 7ff4cd3fc640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a021a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:23.765 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:23.763+0000 7ff4d3408640 -1 rbd::mirror::ImageReplayer: 0x3a66c80 [3/79b55a6a-adb4-4d23-a92b-d5cbf63850f4] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:24.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_25 2026-02-19T20:34:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_25 snap 2026-02-19T20:34:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:34:24.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:24.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_25@snap 2026-02-19T20:34:25.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:25.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_25 2026-02-19T20:34:25.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:25.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:25.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:34:25.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:25.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:25.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_25 2026-02-19T20:34:25.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:25.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:25.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:34:25.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_25 2026-02-19T20:34:25.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_25 2026-02-19T20:34:25.766 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:25.764+0000 7f5fd02ea640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4d01380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:25.766 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:25.764+0000 7f5fd72f8640 -1 rbd::mirror::ImageReplayer: 0x4bd2a00 [3/de88305c-0e9b-4c57-856e-bc3028b13752] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:26.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:26.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:26.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:26.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_26 2026-02-19T20:34:26.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_26 snap 2026-02-19T20:34:26.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:26.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:26.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:34:26.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:26.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_26@snap 2026-02-19T20:34:27.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:27.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_26 2026-02-19T20:34:27.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:27.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:27.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:34:27.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:27.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:27.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_26 2026-02-19T20:34:27.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:27.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:27.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:34:27.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_26 2026-02-19T20:34:27.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_26 2026-02-19T20:34:27.772 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:27.771+0000 7f5fd0aeb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4e5cea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:27.772 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:27.771+0000 7f5fd6af7640 -1 rbd::mirror::ImageReplayer: 0x4a22f00 [3/807691bb-e01b-4205-9b22-0a3c88b5aa24] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:28.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_27 2026-02-19T20:34:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_27 snap 2026-02-19T20:34:28.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:28.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:28.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:34:28.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:28.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_27@snap 2026-02-19T20:34:29.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:29.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_27 2026-02-19T20:34:29.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:29.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:29.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:34:29.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:29.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:29.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_27 2026-02-19T20:34:29.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:29.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:29.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:34:29.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_27 2026-02-19T20:34:29.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_27 2026-02-19T20:34:29.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:29.772+0000 7ff4cd3fc640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a02000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:29.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:29.772+0000 7ff4d3c09640 -1 rbd::mirror::ImageReplayer: 0x3152f00 [3/26aa151c-5c6e-407f-84a4-06e391048d6e] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:30.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:30.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:30.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:30.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_28 2026-02-19T20:34:30.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_28 snap 2026-02-19T20:34:30.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:30.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:30.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:34:30.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:30.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_28@snap 2026-02-19T20:34:31.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:31.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_28 2026-02-19T20:34:31.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:31.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:31.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:34:31.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:31.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:31.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_28 2026-02-19T20:34:31.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:31.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:31.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:34:31.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_28 2026-02-19T20:34:31.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_28 2026-02-19T20:34:31.781 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:31.780+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3815ba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:31.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:31.780+0000 7f0ab18a7640 -1 rbd::mirror::ImageReplayer: 0x1fb9400 [3/0df66c2e-adfc-4e01-b702-4d0a8bd1c719] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:32.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:32.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:32.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:32.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_29 2026-02-19T20:34:32.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_29 snap 2026-02-19T20:34:32.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:32.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:34:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:32.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_29@snap 2026-02-19T20:34:33.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:33.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_29 2026-02-19T20:34:33.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:33.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:33.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:34:33.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:33.702 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:33.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_29 2026-02-19T20:34:33.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:33.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:33.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:34:33.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_29 2026-02-19T20:34:33.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_29 2026-02-19T20:34:33.781 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:33.779+0000 7f5fd02ea640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x41e3a00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:34.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:34.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:34.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:34.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_30 2026-02-19T20:34:34.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_30 snap 2026-02-19T20:34:34.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:34.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:34.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:34:34.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:34.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_30@snap 2026-02-19T20:34:35.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:35.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_30 2026-02-19T20:34:35.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:35.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:35.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:34:35.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:35.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:35.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_30 2026-02-19T20:34:35.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:35.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:35.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:34:35.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_30 2026-02-19T20:34:35.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_30 2026-02-19T20:34:35.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:35.786+0000 7f828ee9a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4082ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:35.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:35.786+0000 7f82956a7640 -1 rbd::mirror::ImageReplayer: 0x1fdbb80 [3/6e1e9027-c0b2-4632-8435-9250281efb2a] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:36.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_31 2026-02-19T20:34:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_31 snap 2026-02-19T20:34:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:34:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:36.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_31@snap 2026-02-19T20:34:37.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:37.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_31 2026-02-19T20:34:37.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:37.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:37.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:34:37.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:37.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:37.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_31 2026-02-19T20:34:37.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:37.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:37.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:34:37.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_31 2026-02-19T20:34:37.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_31 2026-02-19T20:34:37.791 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:37.789+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x381b040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:37.791 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:37.789+0000 7f0ab18a7640 -1 rbd::mirror::ImageReplayer: 0x3704a00 [3/41ae9cd0-a35e-4cfc-8eb3-abb6b3d0a94b] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:38.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:38.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:38.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:38.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_32 2026-02-19T20:34:38.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_32 snap 2026-02-19T20:34:38.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:38.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:38.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:34:38.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:38.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_32@snap 2026-02-19T20:34:39.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:39.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_32 2026-02-19T20:34:39.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:39.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:39.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:34:39.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:39.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:39.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_32 2026-02-19T20:34:39.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:39.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:39.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:34:39.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_32 2026-02-19T20:34:39.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_32 2026-02-19T20:34:39.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:39.790+0000 7f828e699640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4083380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:39.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:39.790+0000 7f82956a7640 -1 rbd::mirror::ImageReplayer: 0x38f2500 [3/8f9355c3-ffc8-4071-aaf4-f8356e2ec162] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:40.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:40.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:40.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:40.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_33 2026-02-19T20:34:40.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_33 snap 2026-02-19T20:34:40.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:40.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:40.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:34:40.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:40.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_33@snap 2026-02-19T20:34:41.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:41.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_33 2026-02-19T20:34:41.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:41.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:41.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:34:41.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:41.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:41.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_33 2026-02-19T20:34:41.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:41.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:41.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:34:41.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_33 2026-02-19T20:34:41.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_33 2026-02-19T20:34:41.800 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:41.798+0000 7ff4cd3fc640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a03380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:41.801 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:34:41.798+0000 7ff4d3c09640 -1 rbd::mirror::ImageReplayer: 0x3a66000 [3/4410c659-b877-49f7-b76c-9e1e17ec8cee] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:42.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:42.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:42.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:42.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_34 2026-02-19T20:34:42.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_34 snap 2026-02-19T20:34:42.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:42.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:42.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:34:42.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:42.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_34@snap 2026-02-19T20:34:43.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:43.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_34 2026-02-19T20:34:43.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:43.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:43.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:34:43.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:43.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:43.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_34 2026-02-19T20:34:43.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:43.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:43.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:34:43.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_34 2026-02-19T20:34:43.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_34 2026-02-19T20:34:43.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:43.797+0000 7f5fd02ea640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x41e3520 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:43.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:43.797+0000 7f5fd72f8640 -1 rbd::mirror::ImageReplayer: 0x2b2d900 [3/e3994cd0-fb3f-4a15-bc7e-aa4c9bd26c87] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:44.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:44.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:44.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:44.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_35 2026-02-19T20:34:44.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_35 snap 2026-02-19T20:34:44.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:44.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:44.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:34:44.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:44.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_35@snap 2026-02-19T20:34:45.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:45.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_35 2026-02-19T20:34:45.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:45.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:45.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:34:45.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:45.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:45.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_35 2026-02-19T20:34:45.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:45.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:45.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:34:45.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_35 2026-02-19T20:34:45.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_35 2026-02-19T20:34:45.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:45.810+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x381b380 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:45.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:45.810+0000 7f0ab18a7640 -1 rbd::mirror::ImageReplayer: 0x1fb9900 [3/1d72d7ff-8dde-42b4-9811-16a6c4bba739] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:46.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:46.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:46.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:46.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_36 2026-02-19T20:34:46.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_36 snap 2026-02-19T20:34:46.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:46.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:46.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:34:46.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:46.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_36@snap 2026-02-19T20:34:47.558 INFO:tasks.ceph.cluster1.osd.0.trial073.stderr:2026-02-19T20:34:47.555+0000 7f635ae8a640 -1 reset not still connected to 0x6af0c30 2026-02-19T20:34:47.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:47.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_36 2026-02-19T20:34:47.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:47.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:47.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:34:47.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:47.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:47.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_36 2026-02-19T20:34:47.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:47.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:47.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:34:47.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_36 2026-02-19T20:34:47.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_36 2026-02-19T20:34:47.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:47.810+0000 7f828e699640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4002000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:48.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:48.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:48.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:48.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_37 2026-02-19T20:34:48.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_37 snap 2026-02-19T20:34:48.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:48.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:48.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:34:48.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:48.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_37@snap 2026-02-19T20:34:49.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:49.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_37 2026-02-19T20:34:49.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:49.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:49.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:34:49.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:49.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:49.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_37 2026-02-19T20:34:49.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:49.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:49.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:34:49.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_37 2026-02-19T20:34:49.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_37 2026-02-19T20:34:49.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:49.704+0000 7f5fd02ea640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x41e3ba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:49.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:49.704+0000 7f5fd72f8640 -1 rbd::mirror::ImageReplayer: 0x4a23b80 [3/a1238f9d-e6e7-407f-88bc-b2822ea2d0df] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:50.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:50.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:50.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:50.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_38 2026-02-19T20:34:50.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_38 snap 2026-02-19T20:34:50.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:50.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:50.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:34:50.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:50.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_38@snap 2026-02-19T20:34:51.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:51.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_38 2026-02-19T20:34:51.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:51.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:51.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:34:51.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:51.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:51.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_38 2026-02-19T20:34:51.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:51.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:51.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:34:51.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_38 2026-02-19T20:34:51.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_38 2026-02-19T20:34:51.704 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:51.702+0000 7f5fd0aeb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4e5d040 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:51.704 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:34:51.702+0000 7f5fd6af7640 -1 rbd::mirror::ImageReplayer: 0x4a23400 [3/bfe65774-7a8a-43f3-a6a6-84525bb3df1b] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:52.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:52.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:52.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:52.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_39 2026-02-19T20:34:52.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_39 snap 2026-02-19T20:34:52.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:52.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:52.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:34:52.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:52.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_39@snap 2026-02-19T20:34:53.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:53.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_39 2026-02-19T20:34:53.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:53.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:53.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:34:53.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:53.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:53.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_39 2026-02-19T20:34:53.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:53.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:53.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:34:53.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_39 2026-02-19T20:34:53.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_39 2026-02-19T20:34:53.716 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:53.714+0000 7f0aaa899640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x2ea41a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:53.716 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:53.714+0000 7f0ab10a6640 -1 rbd::mirror::ImageReplayer: 0x1fb9680 [3/170be547-d7a9-46a7-b6db-d32c6a98434d] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:54.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:54.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:54.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:54.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_40 2026-02-19T20:34:54.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_40 snap 2026-02-19T20:34:54.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:54.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:54.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:34:54.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:54.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_40@snap 2026-02-19T20:34:55.189 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2026-02-19T20:34:55.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2026-02-19T20:34:55.190 INFO:tasks.rbd_mirror_thrash:waiting for 6.027453872941077 secs before reviving daemons 2026-02-19T20:34:55.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:34:55.189+0000 7f8295ea8640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 59241) UID: 1000 2026-02-19T20:34:55.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:55.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_40 2026-02-19T20:34:55.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:55.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:55.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:34:55.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:55.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:55.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_40 2026-02-19T20:34:55.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:55.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:55.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:34:55.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_40 2026-02-19T20:34:55.668 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_40 2026-02-19T20:34:55.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:55.715+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x381b6c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:55.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:55.715+0000 7f0ab18a7640 -1 rbd::mirror::ImageReplayer: 0x1fb9b80 [3/1f232c6e-6e25-4b6b-89c0-3442feb4127a] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:56.216 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:56.214+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x2ea4340 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:56.217 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:56.214+0000 7f0ab18a7640 -1 rbd::mirror::ImageReplayer: 0x1fb9b80 [3/1f232c6e-6e25-4b6b-89c0-3442feb4127a] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:34:56.554 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7823904059826217) > weight (0.25) 2026-02-19T20:34:56.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8020431821540464) > weight (0.25) 2026-02-19T20:34:56.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.44051736270442277) > weight (0.25) 2026-02-19T20:34:56.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5052562266976787) > weight (0.25) 2026-02-19T20:34:56.555 INFO:tasks.rbd_mirror_thrash:waiting for 87.95249702707206 secs before thrashing 2026-02-19T20:34:56.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:56.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:56.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:56.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_41 2026-02-19T20:34:56.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_41 snap 2026-02-19T20:34:56.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:56.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:56.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:34:56.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:56.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_41@snap 2026-02-19T20:34:57.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:57.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_41 2026-02-19T20:34:57.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:57.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:57.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:34:57.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:57.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:57.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_41 2026-02-19T20:34:57.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:57.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:57.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:34:57.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_41 2026-02-19T20:34:57.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_41 2026-02-19T20:34:58.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:34:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:34:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:34:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_42 2026-02-19T20:34:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_42 snap 2026-02-19T20:34:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:58.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:58.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:34:58.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:34:58.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_42@snap 2026-02-19T20:34:59.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:34:59.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_42 2026-02-19T20:34:59.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:59.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:59.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:34:59.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:34:59.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:34:59.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_42 2026-02-19T20:34:59.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:34:59.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:34:59.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:34:59.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_42 2026-02-19T20:34:59.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_42 2026-02-19T20:34:59.722 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:59.719+0000 7f0aaa899640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x2ea4680 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:34:59.723 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:34:59.719+0000 7f0ab10a6640 -1 rbd::mirror::ImageReplayer: 0x3704000 [3/181745d0-d881-4b58-8036-60e15d40ce25] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:35:00.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:35:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_43 2026-02-19T20:35:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_43 snap 2026-02-19T20:35:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:35:00.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:35:00.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_43@snap 2026-02-19T20:35:01.218 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2026-02-19T20:35:01.219 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:35:01.219 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2026-02-19T20:35:01.219 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2026-02-19T20:35:01.219 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2026-02-19T20:35:01.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2026-02-19T20:35:01.222 INFO:tasks.rbd_mirror_thrash:waiting for 118.40207806799346 secs before thrashing 2026-02-19T20:35:01.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:35:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_43 2026-02-19T20:35:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:01.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:01.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:35:01.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:01.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:35:01.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_43 2026-02-19T20:35:01.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:01.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:01.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:35:01.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_43 2026-02-19T20:35:01.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_43 2026-02-19T20:35:01.728 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:35:01.725+0000 7ff4ccbfb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a26ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:35:01.728 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:35:01.725+0000 7ff4d3c09640 -1 rbd::mirror::ImageReplayer: 0x3a66000 [3/786b51f3-eb62-4ad3-80e9-7168fa5f0c2e] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:35:02.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:35:02.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:02.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:02.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_44 2026-02-19T20:35:02.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_44 snap 2026-02-19T20:35:02.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:02.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:02.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:35:02.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:35:02.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_44@snap 2026-02-19T20:35:03.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:35:03.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_44 2026-02-19T20:35:03.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:03.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:03.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:35:03.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:03.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:35:03.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_44 2026-02-19T20:35:03.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:03.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:03.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:35:03.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_44 2026-02-19T20:35:03.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_44 2026-02-19T20:35:03.733 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:35:03.730+0000 7ff4cd3fc640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a02d00 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:35:03.733 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:35:03.730+0000 7ff4d3408640 -1 rbd::mirror::ImageReplayer: 0x3152280 [3/4b187764-f4c1-4c2e-832e-6fc3fbe30766] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:35:04.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:35:04.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:04.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:04.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_45 2026-02-19T20:35:04.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_45 snap 2026-02-19T20:35:04.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:04.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:04.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:35:04.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:35:04.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_45@snap 2026-02-19T20:35:05.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:35:05.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_45 2026-02-19T20:35:05.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:05.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:05.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:35:05.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:05.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:35:05.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_45 2026-02-19T20:35:05.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:05.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:05.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:35:05.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_45 2026-02-19T20:35:05.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_45 2026-02-19T20:35:05.735 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:35:05.733+0000 7f0aaa899640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x2ea4820 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:35:05.735 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:35:05.733+0000 7f0ab10a6640 -1 rbd::mirror::ImageReplayer: 0x1fb9680 [3/11312853-10d6-49e2-a14a-c1c0aeb67353] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:35:06.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:35:06.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:06.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:06.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_46 2026-02-19T20:35:06.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_46 snap 2026-02-19T20:35:06.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:06.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:06.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:35:06.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:35:06.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_46@snap 2026-02-19T20:35:07.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:35:07.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_46 2026-02-19T20:35:07.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:07.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:07.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:35:07.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:07.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:35:07.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_46 2026-02-19T20:35:07.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:07.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:07.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:35:07.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_46 2026-02-19T20:35:07.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_46 2026-02-19T20:35:07.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:35:07.734+0000 7ff4f590f640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x388aea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:35:07.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:35:07.734+0000 7ff4fc91d640 -1 rbd::mirror::ImageReplayer: 0x2195900 [3/fa7f549b-1b88-4438-8fba-afa9db93d973] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:35:07.740 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:35:07.737+0000 7ff4f6110640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x2f9dba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:35:07.740 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:35:07.737+0000 7ff4fc11c640 -1 rbd::mirror::ImageReplayer: 0x2195900 [3/fa7f549b-1b88-4438-8fba-afa9db93d973] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:35:08.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:35:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_47 2026-02-19T20:35:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_47 snap 2026-02-19T20:35:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:08.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:35:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:35:08.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_47@snap 2026-02-19T20:35:09.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:35:09.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_47 2026-02-19T20:35:09.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:09.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:09.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:35:09.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:09.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:35:09.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_47 2026-02-19T20:35:09.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:09.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:09.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:35:09.666 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_47 2026-02-19T20:35:09.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_47 2026-02-19T20:35:09.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:35:09.739+0000 7f0aab09a640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3940000 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:35:09.742 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:35:09.739+0000 7f0ab10a6640 -1 rbd::mirror::ImageReplayer: 0x1fb9b80 [3/9bf92945-bae6-44d1-9712-24406b283d4b] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:35:10.709 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:35:10.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:10.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:10.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_48 2026-02-19T20:35:10.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_48 snap 2026-02-19T20:35:10.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:10.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:10.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:35:10.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:35:10.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_48@snap 2026-02-19T20:35:11.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:35:11.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_48 2026-02-19T20:35:11.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:11.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:11.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:35:11.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:11.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:35:11.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_48 2026-02-19T20:35:11.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:11.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:11.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:35:11.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_48 2026-02-19T20:35:11.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_48 2026-02-19T20:35:11.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:35:11.744+0000 7ff4ccbfb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x3a02ea0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:35:11.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:35:11.744+0000 7ff4d3408640 -1 rbd::mirror::ImageReplayer: 0x3152500 [3/893ecffe-1a73-4473-a8e4-b92236e68f8f] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:35:12.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:35:12.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:12.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:12.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_49 2026-02-19T20:35:12.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_49 snap 2026-02-19T20:35:12.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:12.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:12.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:35:12.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:35:12.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_49@snap 2026-02-19T20:35:13.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:35:13.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_49 2026-02-19T20:35:13.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:13.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:13.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:35:13.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:13.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:35:13.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_49 2026-02-19T20:35:13.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:13.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:13.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:35:13.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_49 2026-02-19T20:35:13.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_49 2026-02-19T20:35:13.750 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:35:13.747+0000 7f5fd0aeb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4e5d6c0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:35:13.750 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:35:13.747+0000 7f5fd6af7640 -1 rbd::mirror::ImageReplayer: 0x4a23400 [3/6b13aa42-62d2-4bf0-b934-8f6d6bb3c8da] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:35:14.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:35:14.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:14.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:14.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_50 2026-02-19T20:35:14.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_snapshot cluster2 mirror image_50 snap 2026-02-19T20:35:14.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:14.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:14.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:35:14.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local snap=snap 2026-02-19T20:35:14.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap rm mirror/image_50@snap 2026-02-19T20:35:15.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing snap: 100% complete...done. 2026-02-19T20:35:15.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_50 2026-02-19T20:35:15.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:15.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:15.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:35:15.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:15.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:35:15.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_50 2026-02-19T20:35:15.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:15.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:15.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:35:15.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_50 2026-02-19T20:35:15.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_50 2026-02-19T20:35:15.750 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:35:15.748+0000 7f5fd0aeb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4e5d860 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:35:16.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 3% complete... Removing image: 6% complete... Removing image: 9% complete... Removing image: 12% complete... Removing image: 15% complete... Removing image: 18% complete... Removing image: 21% complete... Removing image: 25% complete... Removing image: 28% complete... Removing image: 31% complete... Removing image: 34% complete... Removing image: 37% complete... Removing image: 40% complete... Removing image: 43% complete... Removing image: 46% complete... Removing image: 50% complete... Removing image: 53% complete... Removing image: 56% complete... Removing image: 59% complete... Removing image: 62% complete... Removing image: 65% complete... Removing image: 68% complete... Removing image: 71% complete... Removing image: 75% complete... Removing image: 78% complete... Removing image: 81% complete... Removing image: 84% complete... Removing image: 87% complete... Removing image: 90% complete... Removing image: 93% complete... Removing image: 96% complete... Removing image: 100% complete...done. 2026-02-19T20:35:16.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:16.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ testlog 'TEST: image deletions should propagate' 2026-02-19T20:35:16.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ tee -a /tmp/tmp.9BL5XQhnTi/rbd-mirror.test.log 2026-02-19T20:35:16.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ date '+%F %T' 2026-02-19T20:35:16.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo 2026-02-19 20:35:16 TEST: image deletions should propagate 2026-02-19T20:35:16.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:2026-02-19 20:35:16 TEST: image deletions should propagate 2026-02-19T20:35:16.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_pool_images cluster1 mirror 0 2026-02-19T20:35:16.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:16.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:16.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_count=0 2026-02-19T20:35:16.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:16.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local count 2026-02-19T20:35:16.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local last_count=0 2026-02-19T20:35:16.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ true 2026-02-19T20:35:16.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 40 2026-02-19T20:35:16.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in `seq 1 40` 2026-02-19T20:35:16.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 1 -ne 1 2026-02-19T20:35:16.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 -p mirror mirror pool status 2026-02-19T20:35:16.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep 'images: ' 2026-02-19T20:35:16.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cut '-d ' -f 2 2026-02-19T20:35:16.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: failed to query services: (13) Permission denied 2026-02-19T20:35:16.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ count=0 2026-02-19T20:35:16.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 = 0 2026-02-19T20:35:16.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:16.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_pool_healthy cluster1 mirror 0 2026-02-19T20:35:16.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:16.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:16.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:16.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state 2026-02-19T20:35:16.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 40 2026-02-19T20:35:16.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in `seq 1 40` 2026-02-19T20:35:16.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 1 -ne 1 2026-02-19T20:35:16.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster cluster1 -p mirror mirror pool status 2026-02-19T20:35:16.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep 'image health:' 2026-02-19T20:35:16.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cut '-d ' -f 3 2026-02-19T20:35:16.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: failed to query services: (13) Permission denied 2026-02-19T20:35:16.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ state=OK 2026-02-19T20:35:16.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test OK = ERROR 2026-02-19T20:35:16.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test OK = OK 2026-02-19T20:35:16.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:16.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 50 2026-02-19T20:35:16.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:16.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_1 2026-02-19T20:35:16.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_1 deleted 2026-02-19T20:35:16.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_1 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:16.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_1 2026-02-19T20:35:16.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2026-02-19T20:35:16.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:16.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_1: (2) No such file or directory 2026-02-19T20:35:16.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:16.796 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:16.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:16.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_1 deleted '' 2026-02-19T20:35:16.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_1 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_1 2026-02-19T20:35:16.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2026-02-19T20:35:16.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:16.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_1: (2) No such file or directory 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_2 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_2 deleted 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:16.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_2 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_2 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2026-02-19T20:35:16.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:16.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_2: (2) No such file or directory 2026-02-19T20:35:16.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:16.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:16.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:17.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_2 deleted '' 2026-02-19T20:35:17.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:35:17.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:17.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:17.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:17.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_2 2026-02-19T20:35:17.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_2 2026-02-19T20:35:17.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2026-02-19T20:35:17.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_2: (2) No such file or directory 2026-02-19T20:35:17.066 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_3 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_3 deleted 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:35:17.067 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:17.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:17.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_3 2026-02-19T20:35:17.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_3 2026-02-19T20:35:17.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2026-02-19T20:35:17.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_3: (2) No such file or directory 2026-02-19T20:35:17.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:17.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:17.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:17.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_3 deleted '' 2026-02-19T20:35:17.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:35:17.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_3 2026-02-19T20:35:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_3 2026-02-19T20:35:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2026-02-19T20:35:17.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_3: (2) No such file or directory 2026-02-19T20:35:17.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:17.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:17.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:17.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:17.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_4 2026-02-19T20:35:17.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_4 deleted 2026-02-19T20:35:17.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_4 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_4 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2026-02-19T20:35:17.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_4: (2) No such file or directory 2026-02-19T20:35:17.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:17.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:17.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:17.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_4 deleted '' 2026-02-19T20:35:17.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:35:17.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:17.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:17.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_4 2026-02-19T20:35:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_4 2026-02-19T20:35:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2026-02-19T20:35:17.336 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_4: (2) No such file or directory 2026-02-19T20:35:17.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:17.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:17.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:17.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:17.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_5 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_5 deleted 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_5 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_5 2026-02-19T20:35:17.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2026-02-19T20:35:17.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_5: (2) No such file or directory 2026-02-19T20:35:17.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:17.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:17.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:17.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_5 deleted '' 2026-02-19T20:35:17.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_5 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_5 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2026-02-19T20:35:17.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.504 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_5: (2) No such file or directory 2026-02-19T20:35:17.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_6 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_6 deleted 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:35:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:17.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:17.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_6 2026-02-19T20:35:17.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_6 2026-02-19T20:35:17.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2026-02-19T20:35:17.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_6: (2) No such file or directory 2026-02-19T20:35:17.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:17.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:17.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:17.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_6 deleted '' 2026-02-19T20:35:17.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_6 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_6 2026-02-19T20:35:17.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2026-02-19T20:35:17.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_6: (2) No such file or directory 2026-02-19T20:35:17.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:17.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:17.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:17.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:17.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_7 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_7 deleted 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_7 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_7 2026-02-19T20:35:17.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2026-02-19T20:35:17.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:17.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_7: (2) No such file or directory 2026-02-19T20:35:17.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:17.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:17.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:17.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_7 deleted '' 2026-02-19T20:35:17.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:17.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:17.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:35:17.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:17.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:17.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:17.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:17.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_7 2026-02-19T20:35:17.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:17.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:17.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_7 2026-02-19T20:35:17.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2026-02-19T20:35:17.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_7: (2) No such file or directory 2026-02-19T20:35:18.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:18.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:18.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:18.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:18.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_8 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_8 deleted 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_8 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_8 2026-02-19T20:35:18.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2026-02-19T20:35:18.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_8: (2) No such file or directory 2026-02-19T20:35:18.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:18.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:18.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:18.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_8 deleted '' 2026-02-19T20:35:18.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:35:18.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:18.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:18.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:18.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_8 2026-02-19T20:35:18.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_8 2026-02-19T20:35:18.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2026-02-19T20:35:18.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.146 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_8: (2) No such file or directory 2026-02-19T20:35:18.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_9 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_9 deleted 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:18.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_9 2026-02-19T20:35:18.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_9 2026-02-19T20:35:18.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2026-02-19T20:35:18.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_9: (2) No such file or directory 2026-02-19T20:35:18.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:18.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:18.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:18.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_9 deleted '' 2026-02-19T20:35:18.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_9 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_9 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2026-02-19T20:35:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_9: (2) No such file or directory 2026-02-19T20:35:18.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:18.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_10 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_10 deleted 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:18.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_10 2026-02-19T20:35:18.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_10 2026-02-19T20:35:18.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2026-02-19T20:35:18.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_10: (2) No such file or directory 2026-02-19T20:35:18.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:18.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:18.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:18.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_10 deleted '' 2026-02-19T20:35:18.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:35:18.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:18.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:18.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:18.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_10 2026-02-19T20:35:18.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_10 2026-02-19T20:35:18.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2026-02-19T20:35:18.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_10: (2) No such file or directory 2026-02-19T20:35:18.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_11 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_11 deleted 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:18.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_11 2026-02-19T20:35:18.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_11 2026-02-19T20:35:18.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_11 2026-02-19T20:35:18.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_11: (2) No such file or directory 2026-02-19T20:35:18.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:18.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:18.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:18.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_11 deleted '' 2026-02-19T20:35:18.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_11 2026-02-19T20:35:18.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:18.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:18.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:18.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_11 2026-02-19T20:35:18.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_11 2026-02-19T20:35:18.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_11 2026-02-19T20:35:18.567 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_11: (2) No such file or directory 2026-02-19T20:35:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_12 2026-02-19T20:35:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_12 deleted 2026-02-19T20:35:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_12 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_12 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_12 2026-02-19T20:35:18.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_12: (2) No such file or directory 2026-02-19T20:35:18.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:18.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:18.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:18.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_12 deleted '' 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_12 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_12 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_12 2026-02-19T20:35:18.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_12 2026-02-19T20:35:18.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_12: (2) No such file or directory 2026-02-19T20:35:18.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:18.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:18.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:18.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_13 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_13 deleted 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_13 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_13 2026-02-19T20:35:18.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_13 2026-02-19T20:35:18.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_13: (2) No such file or directory 2026-02-19T20:35:18.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:18.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:18.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:18.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_13 deleted '' 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_13 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_13 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_13 2026-02-19T20:35:18.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_13 2026-02-19T20:35:18.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_13: (2) No such file or directory 2026-02-19T20:35:18.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:18.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:18.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:18.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:18.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_14 2026-02-19T20:35:18.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_14 deleted 2026-02-19T20:35:18.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:18.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_14 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_14 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_14 2026-02-19T20:35:18.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:18.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_14: (2) No such file or directory 2026-02-19T20:35:18.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:18.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:18.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:19.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_14 deleted '' 2026-02-19T20:35:19.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_14 2026-02-19T20:35:19.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:19.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:19.002 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:19.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_14 2026-02-19T20:35:19.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_14 2026-02-19T20:35:19.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_14 2026-02-19T20:35:19.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_14: (2) No such file or directory 2026-02-19T20:35:19.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:19.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_15 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_15 deleted 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_15 2026-02-19T20:35:19.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_15 2026-02-19T20:35:19.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_15 2026-02-19T20:35:19.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.047 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_15: (2) No such file or directory 2026-02-19T20:35:19.049 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:19.049 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:19.049 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:19.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_15 deleted '' 2026-02-19T20:35:19.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_15 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_15 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_15 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_15 2026-02-19T20:35:19.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_15: (2) No such file or directory 2026-02-19T20:35:19.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:19.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:19.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_16 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_16 deleted 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_16 2026-02-19T20:35:19.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_16 2026-02-19T20:35:19.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_16 2026-02-19T20:35:19.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_16: (2) No such file or directory 2026-02-19T20:35:19.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:19.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:19.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:19.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_16 deleted '' 2026-02-19T20:35:19.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_16 2026-02-19T20:35:19.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:19.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:19.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:19.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_16 2026-02-19T20:35:19.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_16 2026-02-19T20:35:19.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_16 2026-02-19T20:35:19.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_16: (2) No such file or directory 2026-02-19T20:35:19.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:19.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:19.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:19.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:19.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_17 2026-02-19T20:35:19.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_17 deleted 2026-02-19T20:35:19.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_17 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_17 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_17 2026-02-19T20:35:19.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_17: (2) No such file or directory 2026-02-19T20:35:19.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:19.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:19.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:19.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_17 deleted '' 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_17 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_17 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_17 2026-02-19T20:35:19.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_17 2026-02-19T20:35:19.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_17: (2) No such file or directory 2026-02-19T20:35:19.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:19.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:19.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:19.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_18 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_18 deleted 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_18 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_18 2026-02-19T20:35:19.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_18 2026-02-19T20:35:19.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_18: (2) No such file or directory 2026-02-19T20:35:19.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:19.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:19.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:19.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_18 deleted '' 2026-02-19T20:35:19.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.589 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_18 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_18 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_18 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_18 2026-02-19T20:35:19.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_18: (2) No such file or directory 2026-02-19T20:35:19.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:19.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_19 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_19 deleted 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_19 2026-02-19T20:35:19.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_19 2026-02-19T20:35:19.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_19 2026-02-19T20:35:19.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_19: (2) No such file or directory 2026-02-19T20:35:19.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:19.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:19.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:19.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_19 deleted '' 2026-02-19T20:35:19.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_19 2026-02-19T20:35:19.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:19.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:19.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:19.737 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_19 2026-02-19T20:35:19.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_19 2026-02-19T20:35:19.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_19 2026-02-19T20:35:19.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_19: (2) No such file or directory 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_20 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_20 deleted 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_20 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_20 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_20 2026-02-19T20:35:19.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_20: (2) No such file or directory 2026-02-19T20:35:19.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:19.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:19.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:19.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_20 deleted '' 2026-02-19T20:35:19.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_20 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_20 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_20 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_20 2026-02-19T20:35:19.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_20: (2) No such file or directory 2026-02-19T20:35:19.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_21 2026-02-19T20:35:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_21 deleted 2026-02-19T20:35:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:19.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_21 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_21 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_21 2026-02-19T20:35:19.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:19.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_21: (2) No such file or directory 2026-02-19T20:35:19.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:19.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:19.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:20.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_21 deleted '' 2026-02-19T20:35:20.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_21 2026-02-19T20:35:20.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:20.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:20.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:20.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_21 2026-02-19T20:35:20.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_21 2026-02-19T20:35:20.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_21 2026-02-19T20:35:20.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_21: (2) No such file or directory 2026-02-19T20:35:20.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:20.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:20.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:20.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_22 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_22 deleted 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_22 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_22 2026-02-19T20:35:20.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_22 2026-02-19T20:35:20.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_22: (2) No such file or directory 2026-02-19T20:35:20.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:20.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:20.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:20.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_22 deleted '' 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_22 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_22 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_22 2026-02-19T20:35:20.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_22 2026-02-19T20:35:20.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_22: (2) No such file or directory 2026-02-19T20:35:20.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:20.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:20.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:20.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:20.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_23 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_23 deleted 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_23 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_23 2026-02-19T20:35:20.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_23 2026-02-19T20:35:20.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_23: (2) No such file or directory 2026-02-19T20:35:20.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:20.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:20.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:20.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_23 deleted '' 2026-02-19T20:35:20.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_23 2026-02-19T20:35:20.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:20.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:20.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:20.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_23 2026-02-19T20:35:20.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_23 2026-02-19T20:35:20.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_23 2026-02-19T20:35:20.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_23: (2) No such file or directory 2026-02-19T20:35:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_24 2026-02-19T20:35:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_24 deleted 2026-02-19T20:35:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_24 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_24 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_24 2026-02-19T20:35:20.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_24: (2) No such file or directory 2026-02-19T20:35:20.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:20.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:20.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:20.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_24 deleted '' 2026-02-19T20:35:20.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_24 2026-02-19T20:35:20.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:20.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:20.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:20.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_24 2026-02-19T20:35:20.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_24 2026-02-19T20:35:20.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_24 2026-02-19T20:35:20.468 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_24: (2) No such file or directory 2026-02-19T20:35:20.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:20.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:20.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:20.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:20.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_25 2026-02-19T20:35:20.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_25 deleted 2026-02-19T20:35:20.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:35:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_25 2026-02-19T20:35:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_25 2026-02-19T20:35:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_25 2026-02-19T20:35:20.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_25: (2) No such file or directory 2026-02-19T20:35:20.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:20.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:20.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:20.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_25 deleted '' 2026-02-19T20:35:20.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_25 2026-02-19T20:35:20.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:20.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:20.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:20.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_25 2026-02-19T20:35:20.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_25 2026-02-19T20:35:20.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_25 2026-02-19T20:35:20.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_25: (2) No such file or directory 2026-02-19T20:35:20.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:20.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_26 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_26 deleted 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:20.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_26 2026-02-19T20:35:20.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_26 2026-02-19T20:35:20.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_26 2026-02-19T20:35:20.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_26: (2) No such file or directory 2026-02-19T20:35:20.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:20.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:20.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:20.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_26 deleted '' 2026-02-19T20:35:20.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_26 2026-02-19T20:35:20.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:20.961 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:20.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:20.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_26 2026-02-19T20:35:20.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_26 2026-02-19T20:35:20.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_26 2026-02-19T20:35:20.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:20.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_26: (2) No such file or directory 2026-02-19T20:35:20.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:20.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:20.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:20.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:20.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_27 2026-02-19T20:35:20.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_27 deleted 2026-02-19T20:35:20.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:20.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:20.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:35:20.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:20.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:20.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:20.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:20.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_27 2026-02-19T20:35:20.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:20.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:20.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_27 2026-02-19T20:35:20.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_27 2026-02-19T20:35:20.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_27: (2) No such file or directory 2026-02-19T20:35:21.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:21.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:21.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:21.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_27 deleted '' 2026-02-19T20:35:21.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_27 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_27 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_27 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_27 2026-02-19T20:35:21.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.129 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_27: (2) No such file or directory 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_28 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_28 deleted 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.132 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_28 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_28 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_28 2026-02-19T20:35:21.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_28: (2) No such file or directory 2026-02-19T20:35:21.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:21.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:21.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:21.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_28 deleted '' 2026-02-19T20:35:21.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_28 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_28 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_28 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_28 2026-02-19T20:35:21.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_28: (2) No such file or directory 2026-02-19T20:35:21.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:21.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:21.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:21.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_29 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_29 deleted 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_29 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_29 2026-02-19T20:35:21.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_29 2026-02-19T20:35:21.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_29: (2) No such file or directory 2026-02-19T20:35:21.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:21.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:21.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:21.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_29 deleted '' 2026-02-19T20:35:21.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_29 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_29 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_29 2026-02-19T20:35:21.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_29 2026-02-19T20:35:21.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_29: (2) No such file or directory 2026-02-19T20:35:21.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:21.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:21.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:21.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:21.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_30 2026-02-19T20:35:21.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_30 deleted 2026-02-19T20:35:21.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_30 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_30 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_30 2026-02-19T20:35:21.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_30: (2) No such file or directory 2026-02-19T20:35:21.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:21.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:21.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:21.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_30 deleted '' 2026-02-19T20:35:21.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_30 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_30 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_30 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_30 2026-02-19T20:35:21.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.570 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_30: (2) No such file or directory 2026-02-19T20:35:21.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_31 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_31 deleted 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:21.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_31 2026-02-19T20:35:21.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_31 2026-02-19T20:35:21.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_31 2026-02-19T20:35:21.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_31: (2) No such file or directory 2026-02-19T20:35:21.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:21.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:21.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:21.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_31 deleted '' 2026-02-19T20:35:21.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_31 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_31 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_31 2026-02-19T20:35:21.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_31 2026-02-19T20:35:21.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_31: (2) No such file or directory 2026-02-19T20:35:21.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:21.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_32 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_32 deleted 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:21.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_32 2026-02-19T20:35:21.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_32 2026-02-19T20:35:21.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_32 2026-02-19T20:35:21.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_32: (2) No such file or directory 2026-02-19T20:35:21.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:21.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:21.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:21.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_32 deleted '' 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_32 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_32 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_32 2026-02-19T20:35:21.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_32 2026-02-19T20:35:21.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_32: (2) No such file or directory 2026-02-19T20:35:21.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:21.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:21.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:21.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_33 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_33 deleted 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_33 2026-02-19T20:35:21.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_33 2026-02-19T20:35:21.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_33 2026-02-19T20:35:21.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:21.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_33: (2) No such file or directory 2026-02-19T20:35:21.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:21.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:21.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:21.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_33 deleted '' 2026-02-19T20:35:21.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:21.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:21.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_33 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_33 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_33 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_33 2026-02-19T20:35:21.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:22.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_33: (2) No such file or directory 2026-02-19T20:35:22.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:22.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:22.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:22.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:22.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:22.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_34 2026-02-19T20:35:22.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_34 deleted 2026-02-19T20:35:22.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:22.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_34 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_34 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_34 2026-02-19T20:35:22.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:22.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_34: (2) No such file or directory 2026-02-19T20:35:22.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:22.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:22.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:22.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_34 deleted '' 2026-02-19T20:35:22.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:22.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:22.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_34 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_34 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_34 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_34 2026-02-19T20:35:22.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:22.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_34: (2) No such file or directory 2026-02-19T20:35:22.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:22.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:22.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:22.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:22.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:22.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_35 2026-02-19T20:35:22.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_35 deleted 2026-02-19T20:35:22.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_35 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_35 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_35 2026-02-19T20:35:22.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:22.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_35: (2) No such file or directory 2026-02-19T20:35:22.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:22.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:22.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:22.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_35 deleted '' 2026-02-19T20:35:22.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:22.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:22.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_35 2026-02-19T20:35:22.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:22.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:22.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:22.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:22.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_35 2026-02-19T20:35:22.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:22.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:22.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_35 2026-02-19T20:35:22.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_35 2026-02-19T20:35:22.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:22.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_35: (2) No such file or directory 2026-02-19T20:35:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_36 2026-02-19T20:35:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_36 deleted 2026-02-19T20:35:22.308 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_36 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_36 2026-02-19T20:35:22.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_36 2026-02-19T20:35:22.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:22.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_36: (2) No such file or directory 2026-02-19T20:35:22.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:22.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:22.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:22.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_36 deleted '' 2026-02-19T20:35:22.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:22.432 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_36 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_36 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_36 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_36 2026-02-19T20:35:22.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:22.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_36: (2) No such file or directory 2026-02-19T20:35:22.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:22.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:22.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:22.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:22.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:22.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_37 2026-02-19T20:35:22.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_37 deleted 2026-02-19T20:35:22.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:22.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_37 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_37 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_37 2026-02-19T20:35:22.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:22.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_37: (2) No such file or directory 2026-02-19T20:35:22.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:22.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:22.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:22.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_37 deleted '' 2026-02-19T20:35:22.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:22.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:22.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_37 2026-02-19T20:35:22.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:22.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:22.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:22.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:22.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_37 2026-02-19T20:35:22.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:22.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:22.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_37 2026-02-19T20:35:22.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_37 2026-02-19T20:35:22.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_37: (2) No such file or directory 2026-02-19T20:35:23.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:23.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_38 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_38 deleted 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_38 2026-02-19T20:35:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_38 2026-02-19T20:35:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_38 2026-02-19T20:35:23.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_38: (2) No such file or directory 2026-02-19T20:35:23.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:23.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:23.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:23.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_38 deleted '' 2026-02-19T20:35:23.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_38 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_38 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_38 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_38 2026-02-19T20:35:23.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_38: (2) No such file or directory 2026-02-19T20:35:23.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:23.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.156 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_39 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_39 deleted 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_39 2026-02-19T20:35:23.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_39 2026-02-19T20:35:23.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_39 2026-02-19T20:35:23.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_39: (2) No such file or directory 2026-02-19T20:35:23.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:23.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:23.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:23.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_39 deleted '' 2026-02-19T20:35:23.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_39 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_39 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_39 2026-02-19T20:35:23.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_39 2026-02-19T20:35:23.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_39: (2) No such file or directory 2026-02-19T20:35:23.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:23.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:23.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:23.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:23.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_40 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_40 deleted 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_40 2026-02-19T20:35:23.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_40 2026-02-19T20:35:23.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_40 2026-02-19T20:35:23.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_40: (2) No such file or directory 2026-02-19T20:35:23.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:23.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:23.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:23.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_40 deleted '' 2026-02-19T20:35:23.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_40 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_40 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_40 2026-02-19T20:35:23.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_40 2026-02-19T20:35:23.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_40: (2) No such file or directory 2026-02-19T20:35:23.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:23.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:23.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:23.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:23.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_41 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_41 deleted 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_41 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_41 2026-02-19T20:35:23.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_41 2026-02-19T20:35:23.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_41: (2) No such file or directory 2026-02-19T20:35:23.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:23.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:23.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:23.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_41 deleted '' 2026-02-19T20:35:23.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_41 2026-02-19T20:35:23.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:23.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:23.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:23.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_41 2026-02-19T20:35:23.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_41 2026-02-19T20:35:23.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_41 2026-02-19T20:35:23.574 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_41: (2) No such file or directory 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_42 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_42 deleted 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:35:23.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:23.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:23.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_42 2026-02-19T20:35:23.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_42 2026-02-19T20:35:23.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_42 2026-02-19T20:35:23.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_42: (2) No such file or directory 2026-02-19T20:35:23.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:23.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:23.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:23.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_42 deleted '' 2026-02-19T20:35:23.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_42 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_42 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_42 2026-02-19T20:35:23.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_42 2026-02-19T20:35:23.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_42: (2) No such file or directory 2026-02-19T20:35:23.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:23.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:23.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_43 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_43 deleted 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_43 2026-02-19T20:35:23.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_43 2026-02-19T20:35:23.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_43 2026-02-19T20:35:23.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_43: (2) No such file or directory 2026-02-19T20:35:23.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:23.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:23.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:23.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_43 deleted '' 2026-02-19T20:35:23.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_43 2026-02-19T20:35:23.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:23.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:23.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:23.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_43 2026-02-19T20:35:23.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_43 2026-02-19T20:35:23.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_43 2026-02-19T20:35:23.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_43: (2) No such file or directory 2026-02-19T20:35:23.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:23.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:23.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:23.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:23.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_44 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_44 deleted 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_44 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:23.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:23.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_44 2026-02-19T20:35:23.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_44 2026-02-19T20:35:23.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:23.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_44: (2) No such file or directory 2026-02-19T20:35:23.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:23.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:23.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_44 deleted '' 2026-02-19T20:35:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_44 2026-02-19T20:35:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:24.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_44 2026-02-19T20:35:24.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_44 2026-02-19T20:35:24.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_44 2026-02-19T20:35:24.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_44: (2) No such file or directory 2026-02-19T20:35:24.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:24.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_45 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_45 deleted 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:24.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_45 2026-02-19T20:35:24.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_45 2026-02-19T20:35:24.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_45 2026-02-19T20:35:24.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_45: (2) No such file or directory 2026-02-19T20:35:24.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:24.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:24.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:24.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_45 deleted '' 2026-02-19T20:35:24.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_45 2026-02-19T20:35:24.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:24.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:24.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:24.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_45 2026-02-19T20:35:24.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_45 2026-02-19T20:35:24.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_45 2026-02-19T20:35:24.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_45: (2) No such file or directory 2026-02-19T20:35:24.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:24.174 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_46 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_46 deleted 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:24.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:24.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_46 2026-02-19T20:35:24.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_46 2026-02-19T20:35:24.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_46 2026-02-19T20:35:24.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_46: (2) No such file or directory 2026-02-19T20:35:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:24.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_46 deleted '' 2026-02-19T20:35:24.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_46 2026-02-19T20:35:24.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:24.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:24.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:24.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_46 2026-02-19T20:35:24.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_46 2026-02-19T20:35:24.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_46 2026-02-19T20:35:24.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_46: (2) No such file or directory 2026-02-19T20:35:24.318 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_47 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_47 deleted 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:35:24.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_47 2026-02-19T20:35:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_47 2026-02-19T20:35:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_47 2026-02-19T20:35:24.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_47: (2) No such file or directory 2026-02-19T20:35:24.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:24.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:24.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:24.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_47 deleted '' 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_47 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_47 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_47 2026-02-19T20:35:24.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_47 2026-02-19T20:35:24.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_47: (2) No such file or directory 2026-02-19T20:35:24.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:24.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:24.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:24.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:24.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_48 2026-02-19T20:35:24.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_48 deleted 2026-02-19T20:35:24.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_48 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_48 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_48 2026-02-19T20:35:24.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_48: (2) No such file or directory 2026-02-19T20:35:24.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:24.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:24.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:24.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_48 deleted '' 2026-02-19T20:35:24.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_48 2026-02-19T20:35:24.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:24.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:24.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:24.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_48 2026-02-19T20:35:24.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_48 2026-02-19T20:35:24.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_48 2026-02-19T20:35:24.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_48: (2) No such file or directory 2026-02-19T20:35:24.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:24.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:24.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:24.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:24.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_49 2026-02-19T20:35:24.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_49 deleted 2026-02-19T20:35:24.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:35:24.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:24.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:24.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_49 2026-02-19T20:35:24.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_49 2026-02-19T20:35:24.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_49 2026-02-19T20:35:24.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_49: (2) No such file or directory 2026-02-19T20:35:24.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:24.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:24.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:24.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_49 deleted '' 2026-02-19T20:35:24.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_49 2026-02-19T20:35:24.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:24.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:24.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:24.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_49 2026-02-19T20:35:24.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_49 2026-02-19T20:35:24.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_49 2026-02-19T20:35:24.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_49: (2) No such file or directory 2026-02-19T20:35:24.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:24.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:24.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:24.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 ${IMAGE_COUNT}` 2026-02-19T20:35:24.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_50 2026-02-19T20:35:24.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_50 deleted 2026-02-19T20:35:24.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_50 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_50 2026-02-19T20:35:24.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_50 2026-02-19T20:35:24.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_50: (2) No such file or directory 2026-02-19T20:35:24.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:24.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:24.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:24.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_50 deleted '' 2026-02-19T20:35:24.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_50 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_50 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_50 2026-02-19T20:35:24.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_50 2026-02-19T20:35:24.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:24.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_50: (2) No such file or directory 2026-02-19T20:35:24.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:24.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:24.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:24.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:24.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ testlog 'TEST: delete images during bootstrap' 2026-02-19T20:35:24.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ tee -a /tmp/tmp.9BL5XQhnTi/rbd-mirror.test.log 2026-02-19T20:35:24.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ date '+%F %T' 2026-02-19T20:35:24.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo 2026-02-19 20:35:24 TEST: delete images during bootstrap 2026-02-19T20:35:24.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:2026-02-19 20:35:24 TEST: delete images during bootstrap 2026-02-19T20:35:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_pool_mirror_mode cluster1 mirror image 2026-02-19T20:35:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=image 2026-02-19T20:35:24.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster1 mirror pool enable mirror image 2026-02-19T20:35:24.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd: mirroring is already configured for image mode 2026-02-19T20:35:24.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_pool_mirror_mode cluster2 mirror image 2026-02-19T20:35:24.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:24.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=image 2026-02-19T20:35:24.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror pool enable mirror image 2026-02-19T20:35:24.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd: mirroring is already configured for image mode 2026-02-19T20:35:24.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ start_mirror cluster1 2026-02-19T20:35:24.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:24.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance 2026-02-19T20:35:24.942 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set_cluster_instance cluster1 cluster instance 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local val=cluster1 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster_var_name=cluster 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local instance_var_name=instance 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cluster=cluster1 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=cluster1 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' cluster1 = cluster1 ']' 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ instance=0 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval cluster=cluster1 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ cluster=cluster1 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ eval instance=0 2026-02-19T20:35:24.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ instance=0 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=test 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 1 10 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_1 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_1 512M 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:35:24.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:24.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:35:24.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:35:24.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:35:24.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:35:24.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:35:24.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_1 2026-02-19T20:35:24.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_1 2026-02-19T20:35:24.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:24.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:24.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:24.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:35:24.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_1 snapshot 2026-02-19T20:35:25.634 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:35:25.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_1 2026-02-19T20:35:25.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_1': 2026-02-19T20:35:25.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:35:25.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:35:25.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:35:25.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 191b5aad5c85 2026-02-19T20:35:25.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.191b5aad5c85 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:35:24 2026 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:35:24 2026 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:35:24 2026 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: a9991af2-4de0-4fec-8d09-a5ee604497cc 2026-02-19T20:35:25.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:35:25.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_1 2026-02-19T20:35:25.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:25.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:25.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:25.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:35:25.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:35:25.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:35:25.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_1 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:35:25.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 70739968 bytes 2026-02-19T20:35:25.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 207910912 bytes 2026-02-19T20:35:25.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 218760192 bytes 2026-02-19T20:35:25.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 262057984 bytes 2026-02-19T20:35:26.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 289302528 bytes 2026-02-19T20:35:26.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 365573120 bytes 2026-02-19T20:35:26.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 371527680 bytes 2026-02-19T20:35:26.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 384651264 bytes 2026-02-19T20:35:26.228 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 409730048 bytes 2026-02-19T20:35:26.284 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 443936768 bytes 2026-02-19T20:35:26.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 503368704 bytes 2026-02-19T20:35:26.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 507083776 bytes 2026-02-19T20:35:26.457 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 545665024 bytes 2026-02-19T20:35:26.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 549520384 bytes 2026-02-19T20:35:26.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 550148096 bytes 2026-02-19T20:35:26.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 575077376 bytes 2026-02-19T20:35:26.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 628377600 bytes 2026-02-19T20:35:26.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 649155584 bytes 2026-02-19T20:35:26.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 690585600 bytes 2026-02-19T20:35:26.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 707326976 bytes 2026-02-19T20:35:26.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 750146560 bytes 2026-02-19T20:35:26.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 805321728 bytes 2026-02-19T20:35:26.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 857366528 bytes 2026-02-19T20:35:27.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 865842176 bytes 2026-02-19T20:35:27.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 911623168 bytes 2026-02-19T20:35:27.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 913724416 bytes 2026-02-19T20:35:27.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 914354176 bytes 2026-02-19T20:35:27.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 984516608 bytes 2026-02-19T20:35:27.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1027511296 bytes 2026-02-19T20:35:27.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1076121600 bytes 2026-02-19T20:35:27.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1143068672 bytes 2026-02-19T20:35:27.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1233720320 bytes 2026-02-19T20:35:27.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1277633536 bytes 2026-02-19T20:35:27.596 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1300352000 bytes 2026-02-19T20:35:27.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1356572672 bytes 2026-02-19T20:35:27.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1431835648 bytes 2026-02-19T20:35:27.821 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1486443520 bytes 2026-02-19T20:35:27.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1563340800 bytes 2026-02-19T20:35:27.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1614864384 bytes 2026-02-19T20:35:28.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1658425344 bytes 2026-02-19T20:35:28.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1731536896 bytes 2026-02-19T20:35:28.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1764160512 bytes 2026-02-19T20:35:28.212 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1791110144 bytes 2026-02-19T20:35:28.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1868207104 bytes 2026-02-19T20:35:28.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1894643712 bytes 2026-02-19T20:35:28.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1914730496 bytes 2026-02-19T20:35:28.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1931827200 bytes 2026-02-19T20:35:28.492 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1961222144 bytes 2026-02-19T20:35:28.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1999635456 bytes 2026-02-19T20:35:28.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2020680704 bytes 2026-02-19T20:35:28.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2105353216 bytes 2026-02-19T20:35:28.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2201970688 bytes 2026-02-19T20:35:28.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2237095936 bytes 2026-02-19T20:35:28.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2302880768 bytes 2026-02-19T20:35:28.982 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2366954496 bytes 2026-02-19T20:35:29.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2418108416 bytes 2026-02-19T20:35:29.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2492083200 bytes 2026-02-19T20:35:29.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2532090880 bytes 2026-02-19T20:35:29.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2547603456 bytes 2026-02-19T20:35:29.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2555174912 bytes 2026-02-19T20:35:29.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2608092160 bytes 2026-02-19T20:35:29.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2642283520 bytes 2026-02-19T20:35:29.466 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2669869056 bytes 2026-02-19T20:35:29.522 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2707596288 bytes 2026-02-19T20:35:29.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2735571968 bytes 2026-02-19T20:35:29.674 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2828760064 bytes 2026-02-19T20:35:29.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2852170752 bytes 2026-02-19T20:35:29.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2907646976 bytes 2026-02-19T20:35:29.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2952141824 bytes 2026-02-19T20:35:29.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2994931712 bytes 2026-02-19T20:35:29.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3045498880 bytes 2026-02-19T20:35:30.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3083126784 bytes 2026-02-19T20:35:30.070 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3092830208 bytes 2026-02-19T20:35:30.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3101601792 bytes 2026-02-19T20:35:30.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3161703424 bytes 2026-02-19T20:35:30.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3275923456 bytes 2026-02-19T20:35:30.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3323350016 bytes 2026-02-19T20:35:30.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3361252352 bytes 2026-02-19T20:35:30.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3442542592 bytes 2026-02-19T20:35:30.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3462087680 bytes 2026-02-19T20:35:30.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3530355712 bytes 2026-02-19T20:35:30.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3632395264 bytes 2026-02-19T20:35:30.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3684532224 bytes 2026-02-19T20:35:30.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3709090816 bytes 2026-02-19T20:35:30.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3775482880 bytes 2026-02-19T20:35:30.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3811497984 bytes 2026-02-19T20:35:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3875587072 bytes 2026-02-19T20:35:31.083 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3929744384 bytes 2026-02-19T20:35:31.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3965965312 bytes 2026-02-19T20:35:31.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4028788736 bytes 2026-02-19T20:35:31.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4087995392 bytes 2026-02-19T20:35:31.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4185797632 bytes 2026-02-19T20:35:31.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4272128000 bytes 2026-02-19T20:35:31.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4302046208 bytes 2026-02-19T20:35:31.553 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4343194624 bytes 2026-02-19T20:35:31.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4343838720 bytes 2026-02-19T20:35:31.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4359168000 bytes 2026-02-19T20:35:31.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4408850432 bytes 2026-02-19T20:35:31.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4480141312 bytes 2026-02-19T20:35:31.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4532113408 bytes 2026-02-19T20:35:31.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4560262144 bytes 2026-02-19T20:35:31.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4590068736 bytes 2026-02-19T20:35:32.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4644889600 bytes 2026-02-19T20:35:32.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4693923840 bytes 2026-02-19T20:35:32.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4776633344 bytes 2026-02-19T20:35:32.218 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4789743616 bytes 2026-02-19T20:35:32.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4843852800 bytes 2026-02-19T20:35:32.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4857369600 bytes 2026-02-19T20:35:32.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4872092672 bytes 2026-02-19T20:35:32.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4909839360 bytes 2026-02-19T20:35:32.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4951743488 bytes 2026-02-19T20:35:32.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5005386752 bytes 2026-02-19T20:35:32.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5072488448 bytes 2026-02-19T20:35:32.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5118344192 bytes 2026-02-19T20:35:32.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5159800832 bytes 2026-02-19T20:35:32.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5253767168 bytes 2026-02-19T20:35:32.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5255907328 bytes 2026-02-19T20:35:32.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5317544960 bytes 2026-02-19T20:35:32.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5337462784 bytes 2026-02-19T20:35:33.020 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5375504384 bytes 2026-02-19T20:35:33.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5431306240 bytes 2026-02-19T20:35:33.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5457710080 bytes 2026-02-19T20:35:33.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5527399424 bytes 2026-02-19T20:35:33.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5587612672 bytes 2026-02-19T20:35:33.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5622560768 bytes 2026-02-19T20:35:33.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5649475584 bytes 2026-02-19T20:35:33.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5705475072 bytes 2026-02-19T20:35:33.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5750898688 bytes 2026-02-19T20:35:33.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5788886016 bytes 2026-02-19T20:35:33.620 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5795330048 bytes 2026-02-19T20:35:33.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5840024576 bytes 2026-02-19T20:35:33.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5890981888 bytes 2026-02-19T20:35:33.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5908172800 bytes 2026-02-19T20:35:33.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5942206464 bytes 2026-02-19T20:35:33.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5973916672 bytes 2026-02-19T20:35:33.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6002603008 bytes 2026-02-19T20:35:34.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6077839360 bytes 2026-02-19T20:35:34.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6119506944 bytes 2026-02-19T20:35:34.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6172688384 bytes 2026-02-19T20:35:34.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6183170048 bytes 2026-02-19T20:35:34.252 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6242507776 bytes 2026-02-19T20:35:34.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6266212352 bytes 2026-02-19T20:35:34.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6286576640 bytes 2026-02-19T20:35:34.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6337299456 bytes 2026-02-19T20:35:34.467 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6367413248 bytes 2026-02-19T20:35:34.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6424019968 bytes 2026-02-19T20:35:34.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6424786944 bytes 2026-02-19T20:35:34.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6447939584 bytes 2026-02-19T20:35:34.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6509960192 bytes 2026-02-19T20:35:34.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6566846464 bytes 2026-02-19T20:35:34.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6611501056 bytes 2026-02-19T20:35:34.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6666789888 bytes 2026-02-19T20:35:35.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6863442944 bytes 2026-02-19T20:35:35.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6868600832 bytes 2026-02-19T20:35:35.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6900065280 bytes 2026-02-19T20:35:35.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6947180544 bytes 2026-02-19T20:35:35.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7021577216 bytes 2026-02-19T20:35:35.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7030505472 bytes 2026-02-19T20:35:35.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7077366784 bytes 2026-02-19T20:35:35.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 7172478976 bytes 2026-02-19T20:35:35.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 7223307264 bytes 2026-02-19T20:35:35.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7390610432 bytes 2026-02-19T20:35:35.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7459953664 bytes 2026-02-19T20:35:35.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7548590080 bytes 2026-02-19T20:35:35.928 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7634707456 bytes 2026-02-19T20:35:35.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7670325248 bytes 2026-02-19T20:35:36.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7691043840 bytes 2026-02-19T20:35:36.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7741832192 bytes 2026-02-19T20:35:36.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7805203456 bytes 2026-02-19T20:35:36.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7931088896 bytes 2026-02-19T20:35:36.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7937970176 bytes 2026-02-19T20:35:36.377 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7966330880 bytes 2026-02-19T20:35:36.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 8002909184 bytes 2026-02-19T20:35:36.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 8024926208 bytes 2026-02-19T20:35:36.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 8041893888 bytes 2026-02-19T20:35:36.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 8066525184 bytes 2026-02-19T20:35:36.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 8121375744 bytes 2026-02-19T20:35:36.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 8124303360 bytes 2026-02-19T20:35:36.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 8124931072 bytes 2026-02-19T20:35:36.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 8173215744 bytes 2026-02-19T20:35:36.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 8201476096 bytes 2026-02-19T20:35:36.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 8257419264 bytes 2026-02-19T20:35:36.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 8304777216 bytes 2026-02-19T20:35:36.999 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 8322121728 bytes 2026-02-19T20:35:37.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 8372667392 bytes 2026-02-19T20:35:37.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8424340480 bytes 2026-02-19T20:35:37.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8507974656 bytes 2026-02-19T20:35:37.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8555917312 bytes 2026-02-19T20:35:37.310 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8566974464 bytes 2026-02-19T20:35:37.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8573626368 bytes 2026-02-19T20:35:37.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8610992128 bytes 2026-02-19T20:35:37.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8637811712 bytes 2026-02-19T20:35:37.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8680604672 bytes 2026-02-19T20:35:37.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8746090496 bytes 2026-02-19T20:35:37.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8807598080 bytes 2026-02-19T20:35:37.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8845938688 bytes 2026-02-19T20:35:37.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8874011648 bytes 2026-02-19T20:35:37.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8936673280 bytes 2026-02-19T20:35:37.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8990967808 bytes 2026-02-19T20:35:37.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 9024957440 bytes 2026-02-19T20:35:38.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 9133815808 bytes 2026-02-19T20:35:38.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 9183740928 bytes 2026-02-19T20:35:38.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 9185801216 bytes 2026-02-19T20:35:38.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 9197443072 bytes 2026-02-19T20:35:38.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 9223558144 bytes 2026-02-19T20:35:38.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 9255898112 bytes 2026-02-19T20:35:38.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 9289539584 bytes 2026-02-19T20:35:38.387 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 9305739264 bytes 2026-02-19T20:35:38.450 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 9355749376 bytes 2026-02-19T20:35:38.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 9360137216 bytes 2026-02-19T20:35:38.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 9410918400 bytes 2026-02-19T20:35:38.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 9464239104 bytes 2026-02-19T20:35:38.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 9471017984 bytes 2026-02-19T20:35:38.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_1 present 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:38.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:38.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_1 2026-02-19T20:35:38.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:38.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:38.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_1 2026-02-19T20:35:38.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2026-02-19T20:35:38.671 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:38.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113a367aaff5 2026-02-19T20:35:38.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:38.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:38.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_1 present 113a367aaff5 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113a367aaff5 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_1 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:38.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:38.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_1 2026-02-19T20:35:38.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2026-02-19T20:35:38.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113a367aaff5 2026-02-19T20:35:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113a367aaff5 2026-02-19T20:35:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113a367aaff5 -o 113a367aaff5 = 113a367aaff5 2026-02-19T20:35:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:35:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:35:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_1 2026-02-19T20:35:38.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:38.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:38.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:38.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_1 2026-02-19T20:35:39.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:35:39.628+0000 7f43886f4640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x4f61000 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=185 2026-02-19T20:35:39.631 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:35:39.628+0000 7f43886f4640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x4f61000 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=185, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=b9, last_copied_object_number=128, snap_seqs={185=18446744073709551614}] 2026-02-19T20:35:44.633 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:35:44.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_1 deleted 2026-02-19T20:35:44.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:44.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:44.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:44.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:35:44.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:44.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:35:44.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:44.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_1 2026-02-19T20:35:44.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:44.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:44.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_1 2026-02-19T20:35:44.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2026-02-19T20:35:44.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:44.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_1: (2) No such file or directory 2026-02-19T20:35:44.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:35:44.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:35:44.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:35:44.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_1 deleted '' 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_1 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:35:44.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_1 2026-02-19T20:35:44.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_1 2026-02-19T20:35:44.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:35:44.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_1: (2) No such file or directory 2026-02-19T20:35:44.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:35:44.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:35:44.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:35:44.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:35:44.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_1 2026-02-19T20:35:44.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:44.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:44.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:44.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_1 2026-02-19T20:35:44.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_1 2026-02-19T20:35:44.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:44.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:44.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:44.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:44.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:35:44.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_1 2026-02-19T20:35:44.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:44.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:44.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:44.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_1 2026-02-19T20:35:44.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_1 2026-02-19T20:35:44.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:35:44.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:35:44.864 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:35:44.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:44.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:35:45.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_1 2026-02-19T20:35:45.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:45.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:45.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:45.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_1 2026-02-19T20:35:45.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_1 2026-02-19T20:35:45.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:35:45.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:35:45.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:35:45.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:45.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:35:47.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_1 2026-02-19T20:35:47.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:47.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:47.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:47.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_1 2026-02-19T20:35:47.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_1 2026-02-19T20:35:47.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:35:47.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:35:47.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:35:47.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:47.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:35:51.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_1 2026-02-19T20:35:51.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:35:51.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:35:51.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:35:51.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_1 2026-02-19T20:35:52.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_1 2026-02-19T20:35:52.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:35:52.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:35:52.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:35:52.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:35:52.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:36:00.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_1 2026-02-19T20:36:00.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:00.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:00.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:36:00.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_1 2026-02-19T20:36:00.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_1 2026-02-19T20:36:00.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:36:00.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:36:00.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:36:00.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:36:00.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:36:16.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_1 2026-02-19T20:36:16.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:16.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:16.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_1 2026-02-19T20:36:16.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_1 2026-02-19T20:36:16.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_1 2026-02-19T20:36:16.186 INFO:tasks.ceph.cluster2.osd.0.trial080.stderr:2026-02-19T20:36:16.185+0000 7f5bf91cc640 -1 reset not still connected to 0x23639110 2026-02-19T20:36:16.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:36:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:36:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:36:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_2 2026-02-19T20:36:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_2 512M 2026-02-19T20:36:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:36:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:36:16.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:16.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:36:16.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:36:16.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:36:16.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:36:16.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:36:16.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_2 2026-02-19T20:36:16.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_2 2026-02-19T20:36:16.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:16.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:16.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:16.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:36:16.257 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_2 snapshot 2026-02-19T20:36:16.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:36:16.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_2 2026-02-19T20:36:16.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_2': 2026-02-19T20:36:16.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:36:16.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1951d789b912 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1951d789b912 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:36:16 2026 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:36:16 2026 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:36:16 2026 2026-02-19T20:36:16.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:36:16.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:36:16.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 33a28071-43b9-4ed6-9f38-bfc53b53ff4b 2026-02-19T20:36:16.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:36:16.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_2 2026-02-19T20:36:16.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:16.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:16.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:16.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:36:16.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:36:16.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:36:16.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_2 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:36:16.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 55617536 bytes 2026-02-19T20:36:16.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 122800128 bytes 2026-02-19T20:36:16.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 187688960 bytes 2026-02-19T20:36:16.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 228356096 bytes 2026-02-19T20:36:17.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 265587712 bytes 2026-02-19T20:36:17.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 336649216 bytes 2026-02-19T20:36:17.176 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 356759552 bytes 2026-02-19T20:36:17.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 427727872 bytes 2026-02-19T20:36:17.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 461670400 bytes 2026-02-19T20:36:17.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 481427456 bytes 2026-02-19T20:36:17.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 513356800 bytes 2026-02-19T20:36:17.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 536796160 bytes 2026-02-19T20:36:17.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 575583232 bytes 2026-02-19T20:36:17.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 610183168 bytes 2026-02-19T20:36:17.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 610803712 bytes 2026-02-19T20:36:17.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 642498560 bytes 2026-02-19T20:36:17.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 719105024 bytes 2026-02-19T20:36:17.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 722087936 bytes 2026-02-19T20:36:17.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 726383616 bytes 2026-02-19T20:36:17.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 776538112 bytes 2026-02-19T20:36:17.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 815424512 bytes 2026-02-19T20:36:17.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 841338880 bytes 2026-02-19T20:36:18.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 907983872 bytes 2026-02-19T20:36:18.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 979491840 bytes 2026-02-19T20:36:18.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1019884544 bytes 2026-02-19T20:36:18.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1084016640 bytes 2026-02-19T20:36:18.342 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1180251136 bytes 2026-02-19T20:36:18.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1262526464 bytes 2026-02-19T20:36:18.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1299084288 bytes 2026-02-19T20:36:18.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1334216704 bytes 2026-02-19T20:36:18.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1393337344 bytes 2026-02-19T20:36:18.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1428519936 bytes 2026-02-19T20:36:18.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1476255744 bytes 2026-02-19T20:36:18.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1477361664 bytes 2026-02-19T20:36:18.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1496784896 bytes 2026-02-19T20:36:18.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1581195264 bytes 2026-02-19T20:36:18.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1613073408 bytes 2026-02-19T20:36:19.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1728182272 bytes 2026-02-19T20:36:19.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1748040704 bytes 2026-02-19T20:36:19.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1769034752 bytes 2026-02-19T20:36:19.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1793364992 bytes 2026-02-19T20:36:19.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1807482880 bytes 2026-02-19T20:36:19.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1846610944 bytes 2026-02-19T20:36:19.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1863702528 bytes 2026-02-19T20:36:19.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1886391296 bytes 2026-02-19T20:36:19.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1919734784 bytes 2026-02-19T20:36:19.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1942199296 bytes 2026-02-19T20:36:19.541 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1968955392 bytes 2026-02-19T20:36:19.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2023999488 bytes 2026-02-19T20:36:19.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2074739712 bytes 2026-02-19T20:36:19.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2097924096 bytes 2026-02-19T20:36:19.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2108278784 bytes 2026-02-19T20:36:19.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2160040960 bytes 2026-02-19T20:36:19.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2170196992 bytes 2026-02-19T20:36:19.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2246821888 bytes 2026-02-19T20:36:20.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2305901568 bytes 2026-02-19T20:36:20.094 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2347795456 bytes 2026-02-19T20:36:20.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2379125760 bytes 2026-02-19T20:36:20.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2404105216 bytes 2026-02-19T20:36:20.239 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2441882624 bytes 2026-02-19T20:36:20.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2478408704 bytes 2026-02-19T20:36:20.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2514768896 bytes 2026-02-19T20:36:20.410 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2570261504 bytes 2026-02-19T20:36:20.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2667399168 bytes 2026-02-19T20:36:20.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2674560000 bytes 2026-02-19T20:36:20.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2753139712 bytes 2026-02-19T20:36:20.665 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2784570368 bytes 2026-02-19T20:36:20.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2859538432 bytes 2026-02-19T20:36:20.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2886519808 bytes 2026-02-19T20:36:20.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2941552640 bytes 2026-02-19T20:36:20.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2994795520 bytes 2026-02-19T20:36:21.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3091624960 bytes 2026-02-19T20:36:21.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3126284288 bytes 2026-02-19T20:36:21.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3181448192 bytes 2026-02-19T20:36:21.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3182024704 bytes 2026-02-19T20:36:21.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3222885376 bytes 2026-02-19T20:36:21.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3242292224 bytes 2026-02-19T20:36:21.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3275208704 bytes 2026-02-19T20:36:21.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3321056256 bytes 2026-02-19T20:36:21.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3321692160 bytes 2026-02-19T20:36:21.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3346512896 bytes 2026-02-19T20:36:21.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3397112832 bytes 2026-02-19T20:36:21.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3468127232 bytes 2026-02-19T20:36:21.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3517898752 bytes 2026-02-19T20:36:21.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3578225664 bytes 2026-02-19T20:36:21.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3650720768 bytes 2026-02-19T20:36:21.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3677381632 bytes 2026-02-19T20:36:21.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3710714880 bytes 2026-02-19T20:36:21.970 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3733710848 bytes 2026-02-19T20:36:22.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3772100608 bytes 2026-02-19T20:36:22.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3834844160 bytes 2026-02-19T20:36:22.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3881566208 bytes 2026-02-19T20:36:22.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3935657984 bytes 2026-02-19T20:36:22.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3970198528 bytes 2026-02-19T20:36:22.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4007541760 bytes 2026-02-19T20:36:22.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4059341824 bytes 2026-02-19T20:36:22.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4127398912 bytes 2026-02-19T20:36:22.520 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4131286016 bytes 2026-02-19T20:36:22.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4156606464 bytes 2026-02-19T20:36:22.614 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4184483840 bytes 2026-02-19T20:36:22.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4199553024 bytes 2026-02-19T20:36:22.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4219955200 bytes 2026-02-19T20:36:22.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4245675008 bytes 2026-02-19T20:36:22.776 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4246283264 bytes 2026-02-19T20:36:22.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4292846592 bytes 2026-02-19T20:36:22.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4321932288 bytes 2026-02-19T20:36:22.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4356630528 bytes 2026-02-19T20:36:23.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4401439744 bytes 2026-02-19T20:36:23.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4413851648 bytes 2026-02-19T20:36:23.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4430047232 bytes 2026-02-19T20:36:23.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4503285760 bytes 2026-02-19T20:36:23.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4518788096 bytes 2026-02-19T20:36:23.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4539217920 bytes 2026-02-19T20:36:23.306 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4607444992 bytes 2026-02-19T20:36:23.373 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4633366528 bytes 2026-02-19T20:36:23.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4726788096 bytes 2026-02-19T20:36:23.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4795214848 bytes 2026-02-19T20:36:23.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4829083648 bytes 2026-02-19T20:36:23.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4888337408 bytes 2026-02-19T20:36:23.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4975614976 bytes 2026-02-19T20:36:23.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5072805888 bytes 2026-02-19T20:36:23.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5115805696 bytes 2026-02-19T20:36:23.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5135937536 bytes 2026-02-19T20:36:24.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5160725504 bytes 2026-02-19T20:36:24.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5186598912 bytes 2026-02-19T20:36:24.133 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5219533824 bytes 2026-02-19T20:36:24.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5272784896 bytes 2026-02-19T20:36:24.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5336787968 bytes 2026-02-19T20:36:24.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5360560128 bytes 2026-02-19T20:36:24.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5418087424 bytes 2026-02-19T20:36:24.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5437204480 bytes 2026-02-19T20:36:24.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5509881856 bytes 2026-02-19T20:36:24.508 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5728013105477333) > weight (0.25) 2026-02-19T20:36:24.508 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2026-02-19T20:36:24.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2026-02-19T20:36:24.509 INFO:tasks.rbd_mirror_thrash:waiting for 7.518506381963973 secs before reviving daemons 2026-02-19T20:36:24.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:36:24.508+0000 7f25d29fd640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 54702) UID: 1000 2026-02-19T20:36:24.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5579990016 bytes 2026-02-19T20:36:24.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5614674944 bytes 2026-02-19T20:36:24.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5681200128 bytes 2026-02-19T20:36:24.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5742110720 bytes 2026-02-19T20:36:24.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5768101888 bytes 2026-02-19T20:36:24.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5814806528 bytes 2026-02-19T20:36:24.986 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5868542976 bytes 2026-02-19T20:36:25.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5904747520 bytes 2026-02-19T20:36:25.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5931387904 bytes 2026-02-19T20:36:25.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5951291392 bytes 2026-02-19T20:36:25.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5970094080 bytes 2026-02-19T20:36:25.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5984368640 bytes 2026-02-19T20:36:25.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5989417984 bytes 2026-02-19T20:36:25.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6036552704 bytes 2026-02-19T20:36:25.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6072350720 bytes 2026-02-19T20:36:25.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6120243200 bytes 2026-02-19T20:36:25.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6148186112 bytes 2026-02-19T20:36:25.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6206767104 bytes 2026-02-19T20:36:25.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6259459072 bytes 2026-02-19T20:36:25.701 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6324688896 bytes 2026-02-19T20:36:25.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6350236672 bytes 2026-02-19T20:36:25.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6450080768 bytes 2026-02-19T20:36:25.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6490909696 bytes 2026-02-19T20:36:25.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6494272512 bytes 2026-02-19T20:36:26.017 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6573185024 bytes 2026-02-19T20:36:26.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6659863552 bytes 2026-02-19T20:36:26.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6767736832 bytes 2026-02-19T20:36:26.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6812099584 bytes 2026-02-19T20:36:26.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6830739456 bytes 2026-02-19T20:36:26.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6892296192 bytes 2026-02-19T20:36:26.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6931978240 bytes 2026-02-19T20:36:26.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7072581632 bytes 2026-02-19T20:36:26.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7076859904 bytes 2026-02-19T20:36:26.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7108708352 bytes 2026-02-19T20:36:26.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:36:26.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:36:26.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:36:26.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:36:26.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_2 present 2026-02-19T20:36:26.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:36:26.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:26.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_2 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_2 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2026-02-19T20:36:26.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:36:26.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113a1e1175fc 2026-02-19T20:36:26.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:36:26.711 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:36:26.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_2 present 113a1e1175fc 2026-02-19T20:36:26.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113a1e1175fc 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_2 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_2 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2026-02-19T20:36:26.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:36:26.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113a1e1175fc 2026-02-19T20:36:26.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113a1e1175fc 2026-02-19T20:36:26.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113a1e1175fc -o 113a1e1175fc = 113a1e1175fc 2026-02-19T20:36:26.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:36:26.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:36:26.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:36:26.840 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_2 2026-02-19T20:36:26.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:26.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:26.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:26.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_2 2026-02-19T20:36:27.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:36:27.693+0000 7f43886f4640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x551e400 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=186 2026-02-19T20:36:27.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:36:27.693+0000 7f43886f4640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x551e400 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=186, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=ba, last_copied_object_number=128, snap_seqs={186=18446744073709551614}] 2026-02-19T20:36:32.029 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2026-02-19T20:36:32.029 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:36:32.029 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2026-02-19T20:36:32.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2026-02-19T20:36:32.030 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2026-02-19T20:36:32.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2026-02-19T20:36:32.032 INFO:tasks.rbd_mirror_thrash:waiting for 118.95315861767966 secs before thrashing 2026-02-19T20:36:32.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:36:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_2 deleted 2026-02-19T20:36:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:36:32.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:32.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:32.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:36:32.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:36:32.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:36:32.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:36:32.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_2 2026-02-19T20:36:32.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:36:32.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:36:32.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_2 2026-02-19T20:36:32.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2026-02-19T20:36:32.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:36:32.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_2: (2) No such file or directory 2026-02-19T20:36:32.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:36:32.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:36:32.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:36:32.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_2 deleted '' 2026-02-19T20:36:32.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:36:32.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:32.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:32.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:36:32.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:36:32.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:36:32.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:36:32.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_2 2026-02-19T20:36:32.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:36:32.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:36:32.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_2 2026-02-19T20:36:32.831 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_2 2026-02-19T20:36:32.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:36:32.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_2: (2) No such file or directory 2026-02-19T20:36:32.855 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:36:32.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:36:32.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:36:32.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:36:32.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_2 2026-02-19T20:36:32.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:32.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:32.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:32.856 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_2 2026-02-19T20:36:32.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_2 2026-02-19T20:36:32.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:32.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:32.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:32.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:36:32.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:36:32.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_2 2026-02-19T20:36:32.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:32.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:32.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:32.883 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_2 2026-02-19T20:36:32.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_2 2026-02-19T20:36:32.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:36:32.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:36:32.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:36:32.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:36:32.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:36:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_2 2026-02-19T20:36:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:33.936 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_2 2026-02-19T20:36:33.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_2 2026-02-19T20:36:33.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:36:33.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:36:33.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:36:33.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:36:33.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:36:35.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_2 2026-02-19T20:36:35.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:35.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:35.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:35.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_2 2026-02-19T20:36:36.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_2 2026-02-19T20:36:36.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:36:36.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:36:36.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:36:36.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:36:36.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:36:40.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_2 2026-02-19T20:36:40.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:40.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:40.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:40.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_2 2026-02-19T20:36:40.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_2 2026-02-19T20:36:40.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:36:40.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:36:40.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:36:40.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:36:40.097 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:36:48.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_2 2026-02-19T20:36:48.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:36:48.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:36:48.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:36:48.099 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_2 2026-02-19T20:36:48.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_2 2026-02-19T20:36:48.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:36:48.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:36:48.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:36:48.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:36:48.153 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:36:59.625 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8511462865037785) > weight (0.25) 2026-02-19T20:36:59.625 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8852889475808284) > weight (0.25) 2026-02-19T20:36:59.625 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2026-02-19T20:36:59.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2026-02-19T20:36:59.625 INFO:tasks.rbd_mirror_thrash:waiting for 4.542023527726467 secs before reviving daemons 2026-02-19T20:36:59.626 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:36:59.624+0000 7f0ab20a8640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 72937) UID: 1000 2026-02-19T20:37:04.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_2 2026-02-19T20:37:04.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:04.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:04.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_2 2026-02-19T20:37:04.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_2 2026-02-19T20:37:04.168 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2026-02-19T20:37:04.169 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:37:04.169 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2026-02-19T20:37:04.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2026-02-19T20:37:04.169 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2026-02-19T20:37:04.172 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2026-02-19T20:37:04.172 INFO:tasks.rbd_mirror_thrash:waiting for 98.26358308002825 secs before thrashing 2026-02-19T20:37:04.184 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_2 2026-02-19T20:37:04.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_3 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_3 512M 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:04.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:37:04.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:37:04.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:37:04.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:37:04.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:37:04.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_3 2026-02-19T20:37:04.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_3 2026-02-19T20:37:04.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:37:04.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_3 snapshot 2026-02-19T20:37:04.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:37:04.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_3 2026-02-19T20:37:04.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_3': 2026-02-19T20:37:04.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:37:04.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:37:04.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:37:04.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 199587a6c212 2026-02-19T20:37:04.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.199587a6c212 2026-02-19T20:37:04.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:37:04.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:37:04.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:37:04.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:37:04.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:37:04 2026 2026-02-19T20:37:04.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:37:04 2026 2026-02-19T20:37:04.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:37:04 2026 2026-02-19T20:37:04.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:37:04.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:37:04.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: a6619678-8ac4-4f83-84b5-9c62e22f377a 2026-02-19T20:37:04.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:37:04.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_3 2026-02-19T20:37:04.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:04.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:04.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:04.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:37:04.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=9 2026-02-19T20:37:04.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:37:04.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_3 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:37:04.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 20499456 bytes 2026-02-19T20:37:04.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 85890048 bytes 2026-02-19T20:37:04.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 151094272 bytes 2026-02-19T20:37:04.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 178934784 bytes 2026-02-19T20:37:05.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 221748224 bytes 2026-02-19T20:37:05.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 251644928 bytes 2026-02-19T20:37:05.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 276092928 bytes 2026-02-19T20:37:05.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 331229184 bytes 2026-02-19T20:37:05.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 372221952 bytes 2026-02-19T20:37:05.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 415109120 bytes 2026-02-19T20:37:05.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 432275456 bytes 2026-02-19T20:37:05.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 478910464 bytes 2026-02-19T20:37:05.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 545907712 bytes 2026-02-19T20:37:05.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 624813056 bytes 2026-02-19T20:37:05.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 671773696 bytes 2026-02-19T20:37:05.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 694463488 bytes 2026-02-19T20:37:05.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 731632640 bytes 2026-02-19T20:37:05.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 732318720 bytes 2026-02-19T20:37:05.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 756691968 bytes 2026-02-19T20:37:05.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 793178112 bytes 2026-02-19T20:37:05.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 830988288 bytes 2026-02-19T20:37:05.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 861650944 bytes 2026-02-19T20:37:06.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 924108800 bytes 2026-02-19T20:37:06.096 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 963987456 bytes 2026-02-19T20:37:06.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 986273792 bytes 2026-02-19T20:37:06.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1055716352 bytes 2026-02-19T20:37:06.282 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1094331392 bytes 2026-02-19T20:37:06.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1109790720 bytes 2026-02-19T20:37:06.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1186486272 bytes 2026-02-19T20:37:06.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1217206272 bytes 2026-02-19T20:37:06.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1251700736 bytes 2026-02-19T20:37:06.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1293571072 bytes 2026-02-19T20:37:06.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1338290176 bytes 2026-02-19T20:37:06.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1360608256 bytes 2026-02-19T20:37:06.735 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1394894848 bytes 2026-02-19T20:37:06.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1421982720 bytes 2026-02-19T20:37:06.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1455675392 bytes 2026-02-19T20:37:06.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1507311616 bytes 2026-02-19T20:37:07.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1578615808 bytes 2026-02-19T20:37:07.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1650003968 bytes 2026-02-19T20:37:07.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1672296448 bytes 2026-02-19T20:37:07.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1701898240 bytes 2026-02-19T20:37:07.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1773155328 bytes 2026-02-19T20:37:07.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1799283712 bytes 2026-02-19T20:37:07.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1881192448 bytes 2026-02-19T20:37:07.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1911657472 bytes 2026-02-19T20:37:07.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1982444544 bytes 2026-02-19T20:37:07.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2036730880 bytes 2026-02-19T20:37:07.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2082017280 bytes 2026-02-19T20:37:07.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2110109696 bytes 2026-02-19T20:37:07.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2131510272 bytes 2026-02-19T20:37:07.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2163134464 bytes 2026-02-19T20:37:07.866 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2182530048 bytes 2026-02-19T20:37:07.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2210527232 bytes 2026-02-19T20:37:07.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2290245632 bytes 2026-02-19T20:37:08.041 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2310738944 bytes 2026-02-19T20:37:08.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2405178368 bytes 2026-02-19T20:37:08.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2452546560 bytes 2026-02-19T20:37:08.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2503900160 bytes 2026-02-19T20:37:08.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2572172288 bytes 2026-02-19T20:37:08.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2596296704 bytes 2026-02-19T20:37:08.416 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2601761792 bytes 2026-02-19T20:37:08.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2655286272 bytes 2026-02-19T20:37:08.537 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2681315328 bytes 2026-02-19T20:37:08.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2739228672 bytes 2026-02-19T20:37:08.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2792545280 bytes 2026-02-19T20:37:08.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2793158656 bytes 2026-02-19T20:37:08.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2802605056 bytes 2026-02-19T20:37:08.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2878929920 bytes 2026-02-19T20:37:08.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2906340352 bytes 2026-02-19T20:37:08.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2929507328 bytes 2026-02-19T20:37:08.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2952436736 bytes 2026-02-19T20:37:09.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 2997166080 bytes 2026-02-19T20:37:09.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3024734208 bytes 2026-02-19T20:37:09.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3063122944 bytes 2026-02-19T20:37:09.194 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3091931136 bytes 2026-02-19T20:37:09.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3140692992 bytes 2026-02-19T20:37:09.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3149189120 bytes 2026-02-19T20:37:09.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3149818880 bytes 2026-02-19T20:37:09.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3204458496 bytes 2026-02-19T20:37:09.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3252721664 bytes 2026-02-19T20:37:09.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3253334016 bytes 2026-02-19T20:37:09.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3289905152 bytes 2026-02-19T20:37:09.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3341222912 bytes 2026-02-19T20:37:09.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3412398080 bytes 2026-02-19T20:37:09.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3436995584 bytes 2026-02-19T20:37:09.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3509910528 bytes 2026-02-19T20:37:10.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3554386944 bytes 2026-02-19T20:37:10.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3590453248 bytes 2026-02-19T20:37:10.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3626264576 bytes 2026-02-19T20:37:10.250 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3684140032 bytes 2026-02-19T20:37:10.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3773791232 bytes 2026-02-19T20:37:10.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3826106368 bytes 2026-02-19T20:37:10.451 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3827118080 bytes 2026-02-19T20:37:10.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3876222976 bytes 2026-02-19T20:37:10.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 3891676160 bytes 2026-02-19T20:37:10.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 3914251264 bytes 2026-02-19T20:37:10.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 3933392896 bytes 2026-02-19T20:37:10.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 3973037056 bytes 2026-02-19T20:37:10.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4028138496 bytes 2026-02-19T20:37:10.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4098160640 bytes 2026-02-19T20:37:10.905 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4114520064 bytes 2026-02-19T20:37:10.983 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4197711872 bytes 2026-02-19T20:37:11.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4230311936 bytes 2026-02-19T20:37:11.075 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4234580992 bytes 2026-02-19T20:37:11.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4349557760 bytes 2026-02-19T20:37:11.226 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4373422080 bytes 2026-02-19T20:37:11.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4411742208 bytes 2026-02-19T20:37:11.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4462436352 bytes 2026-02-19T20:37:11.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4534560768 bytes 2026-02-19T20:37:11.448 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4546957312 bytes 2026-02-19T20:37:11.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4600706048 bytes 2026-02-19T20:37:11.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4617742336 bytes 2026-02-19T20:37:11.613 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4668989440 bytes 2026-02-19T20:37:11.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4725054464 bytes 2026-02-19T20:37:11.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4763977728 bytes 2026-02-19T20:37:11.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4770693120 bytes 2026-02-19T20:37:11.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4795790336 bytes 2026-02-19T20:37:11.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4827389952 bytes 2026-02-19T20:37:11.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4864156672 bytes 2026-02-19T20:37:11.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4896419840 bytes 2026-02-19T20:37:12.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4943149056 bytes 2026-02-19T20:37:12.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4954954752 bytes 2026-02-19T20:37:12.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5036490752 bytes 2026-02-19T20:37:12.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5081325568 bytes 2026-02-19T20:37:12.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5110428672 bytes 2026-02-19T20:37:12.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5154321408 bytes 2026-02-19T20:37:12.399 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5189667840 bytes 2026-02-19T20:37:12.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5231010816 bytes 2026-02-19T20:37:12.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5266402304 bytes 2026-02-19T20:37:12.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5288945664 bytes 2026-02-19T20:37:12.673 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5400572928 bytes 2026-02-19T20:37:12.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5437854720 bytes 2026-02-19T20:37:12.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5456472064 bytes 2026-02-19T20:37:12.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5487030272 bytes 2026-02-19T20:37:12.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5565855744 bytes 2026-02-19T20:37:12.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5580985344 bytes 2026-02-19T20:37:13.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5641213952 bytes 2026-02-19T20:37:13.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5700608000 bytes 2026-02-19T20:37:13.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5705878528 bytes 2026-02-19T20:37:13.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5706504192 bytes 2026-02-19T20:37:13.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5715113984 bytes 2026-02-19T20:37:13.251 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5736811520 bytes 2026-02-19T20:37:13.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5767367680 bytes 2026-02-19T20:37:13.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5768165376 bytes 2026-02-19T20:37:13.404 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5842073600 bytes 2026-02-19T20:37:13.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 5864444928 bytes 2026-02-19T20:37:13.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 5870822400 bytes 2026-02-19T20:37:13.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 5916051456 bytes 2026-02-19T20:37:13.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 5953263616 bytes 2026-02-19T20:37:13.653 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 5998932992 bytes 2026-02-19T20:37:13.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:37:13.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:37:13.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_3 present 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:37:13.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_3 2026-02-19T20:37:13.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:37:13.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:37:13.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_3 2026-02-19T20:37:13.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:37:13.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2026-02-19T20:37:13.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113aab41ace0 2026-02-19T20:37:13.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:37:13.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:37:13.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_3 present 113aab41ace0 2026-02-19T20:37:13.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:37:13.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:13.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:13.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:37:13.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113aab41ace0 2026-02-19T20:37:13.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:37:13.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:37:13.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_3 2026-02-19T20:37:13.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:37:13.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:37:13.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_3 2026-02-19T20:37:13.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2026-02-19T20:37:13.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:37:13.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113aab41ace0 2026-02-19T20:37:13.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113aab41ace0 2026-02-19T20:37:13.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113aab41ace0 -o 113aab41ace0 = 113aab41ace0 2026-02-19T20:37:13.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:37:13.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:37:13.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:37:13.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_3 2026-02-19T20:37:13.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:13.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:13.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:13.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_3 2026-02-19T20:37:13.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:37:13.936+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x551e800 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=187 2026-02-19T20:37:13.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:37:13.936+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x551e800 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=187, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=bb, last_copied_object_number=128, snap_seqs={187=18446744073709551614}] 2026-02-19T20:37:18.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:37:18.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_3 deleted 2026-02-19T20:37:18.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:37:18.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:18.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:18.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:37:18.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:37:18.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:37:18.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:37:18.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_3 2026-02-19T20:37:18.944 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:37:18.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:37:18.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_3 2026-02-19T20:37:18.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2026-02-19T20:37:18.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:37:18.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_3: (2) No such file or directory 2026-02-19T20:37:18.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:37:18.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:37:18.968 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_3 deleted '' 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_3 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:37:19.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:37:19.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_3 2026-02-19T20:37:19.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_3 2026-02-19T20:37:19.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:37:19.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_3: (2) No such file or directory 2026-02-19T20:37:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:37:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:37:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:37:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:37:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_3 2026-02-19T20:37:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:19.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_3 2026-02-19T20:37:19.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_3 2026-02-19T20:37:19.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:19.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:19.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:19.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:37:19.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:37:19.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_3 2026-02-19T20:37:19.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:19.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:19.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:19.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_3 2026-02-19T20:37:19.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_3 2026-02-19T20:37:19.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:37:19.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:37:19.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:37:19.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:37:19.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:37:20.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_3 2026-02-19T20:37:20.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:20.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:20.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:20.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_3 2026-02-19T20:37:20.197 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_3 2026-02-19T20:37:20.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:37:20.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:37:20.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:37:20.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:37:20.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:37:22.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_3 2026-02-19T20:37:22.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:22.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:22.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:22.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_3 2026-02-19T20:37:22.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_3 2026-02-19T20:37:22.274 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:37:22.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:37:22.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:37:22.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:37:22.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:37:26.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_3 2026-02-19T20:37:26.277 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:26.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:26.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:26.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_3 2026-02-19T20:37:26.303 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_3 2026-02-19T20:37:26.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:37:26.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:37:26.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:37:26.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:37:26.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:37:34.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_3 2026-02-19T20:37:34.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:34.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:34.330 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:34.331 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_3 2026-02-19T20:37:34.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_3 2026-02-19T20:37:34.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:37:34.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:37:34.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:37:34.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:37:34.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:37:50.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_3 2026-02-19T20:37:50.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:50.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:50.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_3 2026-02-19T20:37:50.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_3 2026-02-19T20:37:50.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_3 2026-02-19T20:37:51.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:37:51.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:37:51.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:37:51.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_4 2026-02-19T20:37:51.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_4 512M 2026-02-19T20:37:51.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:37:51.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_4 2026-02-19T20:37:51.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_4 2026-02-19T20:37:51.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:51.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:51.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:37:51.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:37:51.145 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_4 snapshot 2026-02-19T20:37:51.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:37:51.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_4 2026-02-19T20:37:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_4': 2026-02-19T20:37:51.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:37:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:37:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:37:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 19c74f011ec6 2026-02-19T20:37:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.19c74f011ec6 2026-02-19T20:37:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:37:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:37:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:37:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:37:51.682 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:37:51 2026 2026-02-19T20:37:51.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:37:51 2026 2026-02-19T20:37:51.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:37:51 2026 2026-02-19T20:37:51.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:37:51.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:37:51.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: e3f6daac-71ec-443e-8f4f-3ed4bfc354aa 2026-02-19T20:37:51.683 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:37:51.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_4 2026-02-19T20:37:51.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:37:51.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:37:51.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:37:51.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:37:51.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:37:51.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:37:51.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_4 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:37:51.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 49622016 bytes 2026-02-19T20:37:51.836 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 86497280 bytes 2026-02-19T20:37:51.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 87121920 bytes 2026-02-19T20:37:51.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 238469120 bytes 2026-02-19T20:37:52.059 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 288848896 bytes 2026-02-19T20:37:52.131 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 351711232 bytes 2026-02-19T20:37:52.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 369519616 bytes 2026-02-19T20:37:52.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 392617984 bytes 2026-02-19T20:37:52.276 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 434713600 bytes 2026-02-19T20:37:52.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 465485824 bytes 2026-02-19T20:37:52.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 504818688 bytes 2026-02-19T20:37:52.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 540664832 bytes 2026-02-19T20:37:52.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 621937664 bytes 2026-02-19T20:37:52.772 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 645699584 bytes 2026-02-19T20:37:52.837 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 710921216 bytes 2026-02-19T20:37:52.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 711536640 bytes 2026-02-19T20:37:52.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 729503744 bytes 2026-02-19T20:37:52.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 798044160 bytes 2026-02-19T20:37:53.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 872389632 bytes 2026-02-19T20:37:53.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 908988416 bytes 2026-02-19T20:37:53.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 990446592 bytes 2026-02-19T20:37:53.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1093774336 bytes 2026-02-19T20:37:53.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1149728768 bytes 2026-02-19T20:37:53.422 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1167281152 bytes 2026-02-19T20:37:53.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1267875840 bytes 2026-02-19T20:37:53.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1285251072 bytes 2026-02-19T20:37:53.592 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1294578688 bytes 2026-02-19T20:37:53.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1364478976 bytes 2026-02-19T20:37:53.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1400183808 bytes 2026-02-19T20:37:53.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1453779968 bytes 2026-02-19T20:37:53.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1491342336 bytes 2026-02-19T20:37:53.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1554997248 bytes 2026-02-19T20:37:53.985 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1605680128 bytes 2026-02-19T20:37:54.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1621073920 bytes 2026-02-19T20:37:54.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1642680320 bytes 2026-02-19T20:37:54.161 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1716464640 bytes 2026-02-19T20:37:54.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1761318912 bytes 2026-02-19T20:37:54.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1790920704 bytes 2026-02-19T20:37:54.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1846085632 bytes 2026-02-19T20:37:54.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1903926272 bytes 2026-02-19T20:37:54.460 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1923497984 bytes 2026-02-19T20:37:54.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1939038208 bytes 2026-02-19T20:37:54.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1993742336 bytes 2026-02-19T20:37:54.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2022707200 bytes 2026-02-19T20:37:54.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2067275776 bytes 2026-02-19T20:37:54.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2097290240 bytes 2026-02-19T20:37:54.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2186304512 bytes 2026-02-19T20:37:54.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2214735872 bytes 2026-02-19T20:37:54.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2229790720 bytes 2026-02-19T20:37:55.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2307939328 bytes 2026-02-19T20:37:55.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2314383360 bytes 2026-02-19T20:37:55.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2339994624 bytes 2026-02-19T20:37:55.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2396335104 bytes 2026-02-19T20:37:55.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2444394496 bytes 2026-02-19T20:37:55.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2462513152 bytes 2026-02-19T20:37:55.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2475362304 bytes 2026-02-19T20:37:55.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2496950272 bytes 2026-02-19T20:37:55.395 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2512651264 bytes 2026-02-19T20:37:55.477 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2603140096 bytes 2026-02-19T20:37:55.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2661346304 bytes 2026-02-19T20:37:55.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2694469632 bytes 2026-02-19T20:37:55.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2767620096 bytes 2026-02-19T20:37:55.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2796750848 bytes 2026-02-19T20:37:55.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2821405696 bytes 2026-02-19T20:37:55.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2906263552 bytes 2026-02-19T20:37:55.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2942492672 bytes 2026-02-19T20:37:55.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2975596544 bytes 2026-02-19T20:37:56.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3017746432 bytes 2026-02-19T20:37:56.076 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3022154752 bytes 2026-02-19T20:37:56.134 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3064994816 bytes 2026-02-19T20:37:56.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3091977216 bytes 2026-02-19T20:37:56.225 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3114622976 bytes 2026-02-19T20:37:56.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3163882496 bytes 2026-02-19T20:37:56.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3202649088 bytes 2026-02-19T20:37:56.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3317103616 bytes 2026-02-19T20:37:56.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3337183232 bytes 2026-02-19T20:37:56.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3414392832 bytes 2026-02-19T20:37:56.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3452773376 bytes 2026-02-19T20:37:56.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3495459840 bytes 2026-02-19T20:37:56.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3528891392 bytes 2026-02-19T20:37:56.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3556789248 bytes 2026-02-19T20:37:56.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3647238144 bytes 2026-02-19T20:37:56.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3683200000 bytes 2026-02-19T20:37:57.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3748732928 bytes 2026-02-19T20:37:57.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3791398912 bytes 2026-02-19T20:37:57.148 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3840005120 bytes 2026-02-19T20:37:57.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3907691520 bytes 2026-02-19T20:37:57.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3985200128 bytes 2026-02-19T20:37:57.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 4025949184 bytes 2026-02-19T20:37:57.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 4155450368 bytes 2026-02-19T20:37:57.515 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4165745664 bytes 2026-02-19T20:37:57.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4217756672 bytes 2026-02-19T20:37:57.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4282302464 bytes 2026-02-19T20:37:57.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4350702592 bytes 2026-02-19T20:37:57.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4393160704 bytes 2026-02-19T20:37:57.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4439340032 bytes 2026-02-19T20:37:57.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4482843648 bytes 2026-02-19T20:37:57.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4520838144 bytes 2026-02-19T20:37:58.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4536496128 bytes 2026-02-19T20:37:58.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4587526144 bytes 2026-02-19T20:37:58.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4600781824 bytes 2026-02-19T20:37:58.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4610155520 bytes 2026-02-19T20:37:58.227 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4674874368 bytes 2026-02-19T20:37:58.287 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4720384000 bytes 2026-02-19T20:37:58.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4739062784 bytes 2026-02-19T20:37:58.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4812258304 bytes 2026-02-19T20:37:58.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4835872768 bytes 2026-02-19T20:37:58.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4907189248 bytes 2026-02-19T20:37:58.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4993764352 bytes 2026-02-19T20:37:58.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 5057677312 bytes 2026-02-19T20:37:58.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 5147687936 bytes 2026-02-19T20:37:58.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 5150001152 bytes 2026-02-19T20:37:58.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 5225643008 bytes 2026-02-19T20:37:59.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5352402944 bytes 2026-02-19T20:37:59.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5394148352 bytes 2026-02-19T20:37:59.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5428102144 bytes 2026-02-19T20:37:59.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5457545216 bytes 2026-02-19T20:37:59.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5460243456 bytes 2026-02-19T20:37:59.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5513879552 bytes 2026-02-19T20:37:59.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5564787712 bytes 2026-02-19T20:37:59.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5613272064 bytes 2026-02-19T20:37:59.463 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5657135104 bytes 2026-02-19T20:37:59.535 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5727899648 bytes 2026-02-19T20:37:59.595 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5772572672 bytes 2026-02-19T20:37:59.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5803886592 bytes 2026-02-19T20:37:59.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5856838656 bytes 2026-02-19T20:37:59.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5857871872 bytes 2026-02-19T20:37:59.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5945356288 bytes 2026-02-19T20:37:59.860 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5974090752 bytes 2026-02-19T20:37:59.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 6011420672 bytes 2026-02-19T20:37:59.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 6022466560 bytes 2026-02-19T20:38:00.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 6054685696 bytes 2026-02-19T20:38:00.054 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 6062478336 bytes 2026-02-19T20:38:00.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 6111141888 bytes 2026-02-19T20:38:00.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 6166640640 bytes 2026-02-19T20:38:00.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 6245559296 bytes 2026-02-19T20:38:00.319 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 6269438976 bytes 2026-02-19T20:38:00.379 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 6314668032 bytes 2026-02-19T20:38:00.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6371920896 bytes 2026-02-19T20:38:00.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6399713280 bytes 2026-02-19T20:38:00.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6415575040 bytes 2026-02-19T20:38:00.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6453002240 bytes 2026-02-19T20:38:00.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6523958272 bytes 2026-02-19T20:38:00.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6580037632 bytes 2026-02-19T20:38:00.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6619841536 bytes 2026-02-19T20:38:00.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6649192448 bytes 2026-02-19T20:38:00.914 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6704126976 bytes 2026-02-19T20:38:00.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6704712704 bytes 2026-02-19T20:38:00.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6753128448 bytes 2026-02-19T20:38:01.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6788398080 bytes 2026-02-19T20:38:01.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6825879552 bytes 2026-02-19T20:38:01.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6876132352 bytes 2026-02-19T20:38:01.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6904190976 bytes 2026-02-19T20:38:01.295 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6938251264 bytes 2026-02-19T20:38:01.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6997085184 bytes 2026-02-19T20:38:01.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 7099593728 bytes 2026-02-19T20:38:01.509 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 7146529792 bytes 2026-02-19T20:38:01.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 7165491200 bytes 2026-02-19T20:38:01.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 7215490048 bytes 2026-02-19T20:38:01.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:38:01.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:38:01.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:38:01.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:38:01.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_4 present 2026-02-19T20:38:01.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:38:01.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:01.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_4 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_4 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2026-02-19T20:38:01.691 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:38:01.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113ac0a13ac7 2026-02-19T20:38:01.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:38:01.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:38:01.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_4 present 113ac0a13ac7 2026-02-19T20:38:01.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:38:01.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:01.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:01.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:38:01.819 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113ac0a13ac7 2026-02-19T20:38:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:38:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:38:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_4 2026-02-19T20:38:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:38:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:38:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_4 2026-02-19T20:38:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2026-02-19T20:38:01.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:38:01.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113ac0a13ac7 2026-02-19T20:38:01.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113ac0a13ac7 2026-02-19T20:38:01.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113ac0a13ac7 -o 113ac0a13ac7 = 113ac0a13ac7 2026-02-19T20:38:01.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:38:01.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:38:01.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:38:01.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_4 2026-02-19T20:38:01.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:01.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:01.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:01.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_4 2026-02-19T20:38:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:38:03.016+0000 7f43886f4640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x551e000 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=188 2026-02-19T20:38:03.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:38:03.016+0000 7f43886f4640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x551e000 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=188, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=bc, last_copied_object_number=128, snap_seqs={188=18446744073709551614}] 2026-02-19T20:38:08.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_4 deleted 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_4 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_4 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2026-02-19T20:38:08.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:38:08.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_4: (2) No such file or directory 2026-02-19T20:38:08.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:38:08.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:38:08.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:38:08.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_4 deleted '' 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_4 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_4 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_4 2026-02-19T20:38:08.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:38:08.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_4: (2) No such file or directory 2026-02-19T20:38:08.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:38:08.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:38:08.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:38:08.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:38:08.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_4 2026-02-19T20:38:08.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:08.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:08.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:08.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_4 2026-02-19T20:38:08.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_4 2026-02-19T20:38:08.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:08.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:08.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:08.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:08.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:38:08.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_4 2026-02-19T20:38:08.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:08.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:08.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:08.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_4 2026-02-19T20:38:08.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_4 2026-02-19T20:38:08.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:38:08.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:38:08.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:38:08.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:08.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:38:09.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_4 2026-02-19T20:38:09.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:09.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:09.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:09.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_4 2026-02-19T20:38:09.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_4 2026-02-19T20:38:09.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:38:09.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:38:09.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:38:09.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:09.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:38:11.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_4 2026-02-19T20:38:11.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:11.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:11.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:11.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_4 2026-02-19T20:38:11.341 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_4 2026-02-19T20:38:11.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:38:11.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:38:11.367 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:38:11.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:11.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:38:15.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_4 2026-02-19T20:38:15.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:15.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:15.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:15.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_4 2026-02-19T20:38:15.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_4 2026-02-19T20:38:15.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:38:15.424 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:38:15.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:38:15.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:15.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:38:23.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_4 2026-02-19T20:38:23.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:23.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:23.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:23.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_4 2026-02-19T20:38:23.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_4 2026-02-19T20:38:23.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:38:23.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:38:23.481 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:38:23.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:23.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:38:30.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8642695216458229) > weight (0.25) 2026-02-19T20:38:30.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.456471450841478) > weight (0.25) 2026-02-19T20:38:30.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8287704629720385) > weight (0.25) 2026-02-19T20:38:30.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4998181360852717) > weight (0.25) 2026-02-19T20:38:30.986 INFO:tasks.rbd_mirror_thrash:waiting for 63.98142528437853 secs before thrashing 2026-02-19T20:38:39.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_4 2026-02-19T20:38:39.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:39.484 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_4 2026-02-19T20:38:39.485 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_4 2026-02-19T20:38:39.513 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_4 2026-02-19T20:38:39.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:38:39.621 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:38:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:38:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_5 2026-02-19T20:38:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_5 512M 2026-02-19T20:38:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:38:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:38:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:39.622 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:38:39.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:38:39.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:38:39.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:38:39.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:38:39.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_5 2026-02-19T20:38:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_5 2026-02-19T20:38:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:38:39.652 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_5 snapshot 2026-02-19T20:38:40.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:38:40.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_5 2026-02-19T20:38:40.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_5': 2026-02-19T20:38:40.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:38:40.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:38:40.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:38:40.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 19fd8dfcca2f 2026-02-19T20:38:40.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.19fd8dfcca2f 2026-02-19T20:38:40.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:38:40.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:38:40.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:38:40.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:38:40.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:38:39 2026 2026-02-19T20:38:40.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:38:39 2026 2026-02-19T20:38:40.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:38:39 2026 2026-02-19T20:38:40.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:38:40.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:38:40.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 426d263f-e27b-4857-b93a-1fab2c1e4d09 2026-02-19T20:38:40.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:38:40.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_5 2026-02-19T20:38:40.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:40.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:40.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:40.688 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:38:40.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=10 2026-02-19T20:38:40.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:38:40.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_5 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:38:40.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 90165248 bytes 2026-02-19T20:38:40.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 165910528 bytes 2026-02-19T20:38:40.943 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 206070784 bytes 2026-02-19T20:38:41.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 259183616 bytes 2026-02-19T20:38:41.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 282647552 bytes 2026-02-19T20:38:41.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 330784768 bytes 2026-02-19T20:38:41.165 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 368374784 bytes 2026-02-19T20:38:41.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 499806208 bytes 2026-02-19T20:38:41.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 529399808 bytes 2026-02-19T20:38:41.374 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 580685824 bytes 2026-02-19T20:38:41.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 624936960 bytes 2026-02-19T20:38:41.526 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 698342400 bytes 2026-02-19T20:38:41.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 775664640 bytes 2026-02-19T20:38:41.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 833196032 bytes 2026-02-19T20:38:41.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 853236736 bytes 2026-02-19T20:38:41.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 895031296 bytes 2026-02-19T20:38:41.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 970412032 bytes 2026-02-19T20:38:41.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 1023105024 bytes 2026-02-19T20:38:42.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 1100170240 bytes 2026-02-19T20:38:42.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 1117970432 bytes 2026-02-19T20:38:42.102 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1136719872 bytes 2026-02-19T20:38:42.149 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1166467072 bytes 2026-02-19T20:38:42.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1167806464 bytes 2026-02-19T20:38:42.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1245684736 bytes 2026-02-19T20:38:42.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1297394688 bytes 2026-02-19T20:38:42.394 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1353437184 bytes 2026-02-19T20:38:42.437 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9321455615968284) > weight (0.25) 2026-02-19T20:38:42.437 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2026-02-19T20:38:42.438 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2026-02-19T20:38:42.438 INFO:tasks.rbd_mirror_thrash:waiting for 7.8474279289695845 secs before reviving daemons 2026-02-19T20:38:42.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:38:42.436+0000 7ff4d440a640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 60238) UID: 1000 2026-02-19T20:38:42.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1401149440 bytes 2026-02-19T20:38:42.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1446885376 bytes 2026-02-19T20:38:42.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1455898624 bytes 2026-02-19T20:38:42.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1468468224 bytes 2026-02-19T20:38:42.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1482920960 bytes 2026-02-19T20:38:42.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1511997440 bytes 2026-02-19T20:38:42.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1542050816 bytes 2026-02-19T20:38:42.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1572339712 bytes 2026-02-19T20:38:42.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1604077568 bytes 2026-02-19T20:38:42.898 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1634635776 bytes 2026-02-19T20:38:42.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1703395328 bytes 2026-02-19T20:38:43.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1731238912 bytes 2026-02-19T20:38:43.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1815824384 bytes 2026-02-19T20:38:43.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1838168064 bytes 2026-02-19T20:38:43.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1885918208 bytes 2026-02-19T20:38:43.264 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1906238464 bytes 2026-02-19T20:38:43.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1965261824 bytes 2026-02-19T20:38:43.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2029573120 bytes 2026-02-19T20:38:43.487 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2099105792 bytes 2026-02-19T20:38:43.540 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2123800576 bytes 2026-02-19T20:38:43.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2166868992 bytes 2026-02-19T20:38:43.667 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2221528064 bytes 2026-02-19T20:38:43.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2228065280 bytes 2026-02-19T20:38:43.752 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2263411712 bytes 2026-02-19T20:38:43.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2295675904 bytes 2026-02-19T20:38:43.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2350652416 bytes 2026-02-19T20:38:43.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2434363392 bytes 2026-02-19T20:38:43.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2442572800 bytes 2026-02-19T20:38:44.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2486389760 bytes 2026-02-19T20:38:44.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2502737920 bytes 2026-02-19T20:38:44.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2529373184 bytes 2026-02-19T20:38:44.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2537456640 bytes 2026-02-19T20:38:44.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2622433280 bytes 2026-02-19T20:38:44.317 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2638912512 bytes 2026-02-19T20:38:44.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2664057856 bytes 2026-02-19T20:38:44.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2674185216 bytes 2026-02-19T20:38:44.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2692194304 bytes 2026-02-19T20:38:44.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2742738944 bytes 2026-02-19T20:38:44.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2820761600 bytes 2026-02-19T20:38:44.663 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2896713728 bytes 2026-02-19T20:38:44.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2897366016 bytes 2026-02-19T20:38:44.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2905786368 bytes 2026-02-19T20:38:44.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2939923456 bytes 2026-02-19T20:38:44.829 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2954320896 bytes 2026-02-19T20:38:44.933 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3064205312 bytes 2026-02-19T20:38:45.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3141669888 bytes 2026-02-19T20:38:45.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3185320960 bytes 2026-02-19T20:38:45.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3267923968 bytes 2026-02-19T20:38:45.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3280710656 bytes 2026-02-19T20:38:45.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3358385152 bytes 2026-02-19T20:38:45.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3359336448 bytes 2026-02-19T20:38:45.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3421636608 bytes 2026-02-19T20:38:45.446 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3477220352 bytes 2026-02-19T20:38:45.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3545305088 bytes 2026-02-19T20:38:45.586 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3607209984 bytes 2026-02-19T20:38:45.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3683012608 bytes 2026-02-19T20:38:45.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3752087552 bytes 2026-02-19T20:38:45.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3780259840 bytes 2026-02-19T20:38:45.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3800141824 bytes 2026-02-19T20:38:45.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3810894848 bytes 2026-02-19T20:38:45.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3822824448 bytes 2026-02-19T20:38:45.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3849602048 bytes 2026-02-19T20:38:45.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3858551808 bytes 2026-02-19T20:38:46.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3914637312 bytes 2026-02-19T20:38:46.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4014573568 bytes 2026-02-19T20:38:46.396 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4057142272 bytes 2026-02-19T20:38:46.434 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4066236416 bytes 2026-02-19T20:38:46.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4116333568 bytes 2026-02-19T20:38:46.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4143509504 bytes 2026-02-19T20:38:46.575 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4144132096 bytes 2026-02-19T20:38:46.604 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4144750592 bytes 2026-02-19T20:38:46.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4145357824 bytes 2026-02-19T20:38:46.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4166322176 bytes 2026-02-19T20:38:46.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4219667456 bytes 2026-02-19T20:38:46.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4235217920 bytes 2026-02-19T20:38:46.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4237116416 bytes 2026-02-19T20:38:46.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4335593472 bytes 2026-02-19T20:38:46.990 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4412384256 bytes 2026-02-19T20:38:47.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4445615104 bytes 2026-02-19T20:38:47.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4498838528 bytes 2026-02-19T20:38:47.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4557094912 bytes 2026-02-19T20:38:47.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4577475584 bytes 2026-02-19T20:38:47.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4608253952 bytes 2026-02-19T20:38:47.347 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4625358848 bytes 2026-02-19T20:38:47.401 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4662354944 bytes 2026-02-19T20:38:47.475 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4730292224 bytes 2026-02-19T20:38:47.531 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4768963584 bytes 2026-02-19T20:38:47.591 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4818842624 bytes 2026-02-19T20:38:47.638 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4834478080 bytes 2026-02-19T20:38:47.710 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4887023616 bytes 2026-02-19T20:38:47.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4961136640 bytes 2026-02-19T20:38:47.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4993641472 bytes 2026-02-19T20:38:47.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5000634368 bytes 2026-02-19T20:38:47.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5023209472 bytes 2026-02-19T20:38:47.988 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5090937856 bytes 2026-02-19T20:38:48.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5091553280 bytes 2026-02-19T20:38:48.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5122191360 bytes 2026-02-19T20:38:48.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5186681856 bytes 2026-02-19T20:38:48.206 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5223772160 bytes 2026-02-19T20:38:48.263 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5272006656 bytes 2026-02-19T20:38:48.362 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5386824704 bytes 2026-02-19T20:38:48.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5412728832 bytes 2026-02-19T20:38:48.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5437862912 bytes 2026-02-19T20:38:48.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5515581440 bytes 2026-02-19T20:38:48.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5554964480 bytes 2026-02-19T20:38:48.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5655046144 bytes 2026-02-19T20:38:48.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5696526336 bytes 2026-02-19T20:38:48.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5706274816 bytes 2026-02-19T20:38:49.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5731782656 bytes 2026-02-19T20:38:49.128 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5783413760 bytes 2026-02-19T20:38:49.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5827789824 bytes 2026-02-19T20:38:49.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5917542400 bytes 2026-02-19T20:38:49.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5927736320 bytes 2026-02-19T20:38:49.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5972727808 bytes 2026-02-19T20:38:49.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6004183040 bytes 2026-02-19T20:38:49.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6043031552 bytes 2026-02-19T20:38:49.569 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6115999744 bytes 2026-02-19T20:38:49.628 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6163299328 bytes 2026-02-19T20:38:49.677 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6191873024 bytes 2026-02-19T20:38:49.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6213023744 bytes 2026-02-19T20:38:49.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6251319296 bytes 2026-02-19T20:38:49.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6266625024 bytes 2026-02-19T20:38:49.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6296706048 bytes 2026-02-19T20:38:49.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6359178240 bytes 2026-02-19T20:38:50.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6383164416 bytes 2026-02-19T20:38:50.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6416505856 bytes 2026-02-19T20:38:50.103 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6440671232 bytes 2026-02-19T20:38:50.211 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6551092224 bytes 2026-02-19T20:38:50.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6570629120 bytes 2026-02-19T20:38:50.286 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2026-02-19T20:38:50.286 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:38:50.286 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2026-02-19T20:38:50.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2026-02-19T20:38:50.287 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2026-02-19T20:38:50.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2026-02-19T20:38:50.290 INFO:tasks.rbd_mirror_thrash:waiting for 107.39689016994646 secs before thrashing 2026-02-19T20:38:50.329 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6615973888 bytes 2026-02-19T20:38:50.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6676135936 bytes 2026-02-19T20:38:50.493 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6748647424 bytes 2026-02-19T20:38:50.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6798489600 bytes 2026-02-19T20:38:50.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6842428416 bytes 2026-02-19T20:38:50.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6876361728 bytes 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_5 present 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_5 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:38:50.694 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:38:50.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_5 2026-02-19T20:38:50.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2026-02-19T20:38:50.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:38:50.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113ac7d9300c 2026-02-19T20:38:50.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:38:50.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:38:50.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_5 present 113ac7d9300c 2026-02-19T20:38:50.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:38:50.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:50.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:50.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:38:50.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113ac7d9300c 2026-02-19T20:38:50.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:38:50.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:38:50.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_5 2026-02-19T20:38:50.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:38:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:38:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_5 2026-02-19T20:38:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2026-02-19T20:38:50.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:38:50.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113ac7d9300c 2026-02-19T20:38:50.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113ac7d9300c 2026-02-19T20:38:50.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113ac7d9300c -o 113ac7d9300c = 113ac7d9300c 2026-02-19T20:38:50.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:38:50.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:38:50.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:38:50.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_5 2026-02-19T20:38:50.852 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:50.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:50.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:50.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_5 2026-02-19T20:38:51.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:38:51.418+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x3e68c00 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=189 2026-02-19T20:38:51.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:38:51.418+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x3e68c00 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=189, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=bd, last_copied_object_number=128, snap_seqs={189=18446744073709551614}] 2026-02-19T20:38:55.338 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:38:55.336+0000 7f5fd0aeb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4da21a0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:38:55.339 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:38:55.336+0000 7f5fd72f8640 -1 rbd::mirror::ImageReplayer: 0x4a23400 [3/426d263f-e27b-4857-b93a-1fab2c1e4d09] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:38:56.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:38:56.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_5 deleted 2026-02-19T20:38:56.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:38:56.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:56.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:56.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:38:56.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:38:56.429 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:38:56.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:38:56.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_5 2026-02-19T20:38:56.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:38:56.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:38:56.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_5 2026-02-19T20:38:56.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2026-02-19T20:38:56.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:38:56.452 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_5: (2) No such file or directory 2026-02-19T20:38:56.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:38:56.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:38:56.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:38:56.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_5 deleted '' 2026-02-19T20:38:56.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_5 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:38:56.557 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_5 2026-02-19T20:38:56.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_5 2026-02-19T20:38:56.558 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:38:56.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_5: (2) No such file or directory 2026-02-19T20:38:56.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:38:56.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:38:56.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:38:56.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:38:56.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_5 2026-02-19T20:38:56.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:56.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:56.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:56.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_5 2026-02-19T20:38:56.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_5 2026-02-19T20:38:56.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:56.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:56.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:56.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:56.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:38:56.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_5 2026-02-19T20:38:56.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:56.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:56.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:56.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_5 2026-02-19T20:38:56.631 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_5 2026-02-19T20:38:56.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:38:56.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:38:56.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:38:56.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:56.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:38:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_5 2026-02-19T20:38:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:57.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_5 2026-02-19T20:38:57.684 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_5 2026-02-19T20:38:57.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:38:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:38:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:38:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:57.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:38:59.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_5 2026-02-19T20:38:59.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:38:59.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:38:59.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:38:59.715 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_5 2026-02-19T20:38:59.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_5 2026-02-19T20:38:59.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:38:59.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:38:59.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:38:59.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:38:59.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:39:03.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_5 2026-02-19T20:39:03.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:03.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:03.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:39:03.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_5 2026-02-19T20:39:03.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_5 2026-02-19T20:39:03.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:39:03.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:39:03.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:39:03.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:39:03.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:39:11.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_5 2026-02-19T20:39:11.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:11.825 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:11.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:39:11.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_5 2026-02-19T20:39:11.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_5 2026-02-19T20:39:11.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:39:11.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:39:11.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:39:11.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:39:11.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:39:27.881 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_5 2026-02-19T20:39:27.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:27.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:27.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_5 2026-02-19T20:39:27.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_5 2026-02-19T20:39:27.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_5 2026-02-19T20:39:28.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:39:28.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:39:28.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:39:28.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_6 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_6 512M 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:39:28.015 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_6 2026-02-19T20:39:28.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_6 2026-02-19T20:39:28.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:28.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:28.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:28.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:39:28.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_6 snapshot 2026-02-19T20:39:28.714 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:39:28.718 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_6 2026-02-19T20:39:28.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_6': 2026-02-19T20:39:28.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:39:28.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:39:28.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:39:28.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1a3cbe60d7b4 2026-02-19T20:39:28.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1a3cbe60d7b4 2026-02-19T20:39:28.741 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:39:28.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:39:28.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:39:28.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:39:28.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:39:28 2026 2026-02-19T20:39:28.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:39:28 2026 2026-02-19T20:39:28.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:39:28 2026 2026-02-19T20:39:28.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:39:28.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:39:28.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: a57a2419-2b37-4c34-83be-a2e72a49d14b 2026-02-19T20:39:28.743 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:39:28.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_6 2026-02-19T20:39:28.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:28.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:28.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:28.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:39:28.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=11 2026-02-19T20:39:28.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:39:28.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_6 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:39:28.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 16159744 bytes 2026-02-19T20:39:28.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 44239872 bytes 2026-02-19T20:39:28.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 58650624 bytes 2026-02-19T20:39:28.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 96335872 bytes 2026-02-19T20:39:29.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 129373184 bytes 2026-02-19T20:39:29.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 185701376 bytes 2026-02-19T20:39:29.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 199681024 bytes 2026-02-19T20:39:29.150 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 222288896 bytes 2026-02-19T20:39:29.208 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 254794752 bytes 2026-02-19T20:39:29.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 289188864 bytes 2026-02-19T20:39:29.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 372997120 bytes 2026-02-19T20:39:29.392 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 381006848 bytes 2026-02-19T20:39:29.442 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 423478272 bytes 2026-02-19T20:39:29.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 424093696 bytes 2026-02-19T20:39:29.551 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 484897792 bytes 2026-02-19T20:39:29.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 522677248 bytes 2026-02-19T20:39:29.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 542635008 bytes 2026-02-19T20:39:29.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 556155904 bytes 2026-02-19T20:39:29.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 581977088 bytes 2026-02-19T20:39:29.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 642143232 bytes 2026-02-19T20:39:29.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 731606016 bytes 2026-02-19T20:39:29.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 782446592 bytes 2026-02-19T20:39:30.023 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 805385216 bytes 2026-02-19T20:39:30.061 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 821823488 bytes 2026-02-19T20:39:30.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 925928448 bytes 2026-02-19T20:39:30.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1023124480 bytes 2026-02-19T20:39:30.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1081013248 bytes 2026-02-19T20:39:30.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1095298048 bytes 2026-02-19T20:39:30.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1158019072 bytes 2026-02-19T20:39:30.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1176569856 bytes 2026-02-19T20:39:30.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1226397696 bytes 2026-02-19T20:39:30.636 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1339433984 bytes 2026-02-19T20:39:30.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1356380160 bytes 2026-02-19T20:39:30.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1392602112 bytes 2026-02-19T20:39:30.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1440348160 bytes 2026-02-19T20:39:30.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1440969728 bytes 2026-02-19T20:39:30.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1504291840 bytes 2026-02-19T20:39:30.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1535411200 bytes 2026-02-19T20:39:31.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1564981248 bytes 2026-02-19T20:39:31.068 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1604590592 bytes 2026-02-19T20:39:31.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1629952000 bytes 2026-02-19T20:39:31.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1673879552 bytes 2026-02-19T20:39:31.217 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1679614976 bytes 2026-02-19T20:39:31.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1727604736 bytes 2026-02-19T20:39:31.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1762646016 bytes 2026-02-19T20:39:31.426 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1844564992 bytes 2026-02-19T20:39:31.511 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 1912563712 bytes 2026-02-19T20:39:31.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 1931770880 bytes 2026-02-19T20:39:31.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 1988402176 bytes 2026-02-19T20:39:31.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2022989824 bytes 2026-02-19T20:39:31.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2073395200 bytes 2026-02-19T20:39:31.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2109436928 bytes 2026-02-19T20:39:31.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2134076416 bytes 2026-02-19T20:39:31.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2195860480 bytes 2026-02-19T20:39:31.956 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2225581056 bytes 2026-02-19T20:39:32.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2273895424 bytes 2026-02-19T20:39:32.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2375665664 bytes 2026-02-19T20:39:32.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2413468672 bytes 2026-02-19T20:39:32.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2415166464 bytes 2026-02-19T20:39:32.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2482024448 bytes 2026-02-19T20:39:32.334 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2517914624 bytes 2026-02-19T20:39:32.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2554517504 bytes 2026-02-19T20:39:32.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2595720192 bytes 2026-02-19T20:39:32.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2725175296 bytes 2026-02-19T20:39:32.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2747576320 bytes 2026-02-19T20:39:32.679 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2811287552 bytes 2026-02-19T20:39:32.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2860300288 bytes 2026-02-19T20:39:32.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2920761344 bytes 2026-02-19T20:39:32.873 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2955580416 bytes 2026-02-19T20:39:32.917 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2967118848 bytes 2026-02-19T20:39:32.977 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3012371456 bytes 2026-02-19T20:39:33.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3016752128 bytes 2026-02-19T20:39:33.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3036980224 bytes 2026-02-19T20:39:33.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3046818816 bytes 2026-02-19T20:39:33.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3065985024 bytes 2026-02-19T20:39:33.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3164346368 bytes 2026-02-19T20:39:33.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3195041792 bytes 2026-02-19T20:39:33.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3297167360 bytes 2026-02-19T20:39:33.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3375556608 bytes 2026-02-19T20:39:33.523 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3397808128 bytes 2026-02-19T20:39:33.627 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3513081856 bytes 2026-02-19T20:39:33.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3567964160 bytes 2026-02-19T20:39:33.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3594999808 bytes 2026-02-19T20:39:33.778 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3604254720 bytes 2026-02-19T20:39:33.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3618124800 bytes 2026-02-19T20:39:33.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3673282560 bytes 2026-02-19T20:39:33.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3741554688 bytes 2026-02-19T20:39:34.013 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3775020032 bytes 2026-02-19T20:39:34.086 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3811939328 bytes 2026-02-19T20:39:34.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3864725504 bytes 2026-02-19T20:39:34.196 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3889723392 bytes 2026-02-19T20:39:34.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3935296512 bytes 2026-02-19T20:39:34.315 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3961792512 bytes 2026-02-19T20:39:34.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4015772672 bytes 2026-02-19T20:39:34.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4040863744 bytes 2026-02-19T20:39:34.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4115088384 bytes 2026-02-19T20:39:34.571 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4126794752 bytes 2026-02-19T20:39:34.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4155155456 bytes 2026-02-19T20:39:34.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4195791872 bytes 2026-02-19T20:39:34.717 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4216162304 bytes 2026-02-19T20:39:34.811 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4294844416 bytes 2026-02-19T20:39:34.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4339222528 bytes 2026-02-19T20:39:34.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4382269440 bytes 2026-02-19T20:39:34.968 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2026-02-19T20:39:34.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2026-02-19T20:39:34.969 INFO:tasks.rbd_mirror_thrash:waiting for 4.137503630542283 secs before reviving daemons 2026-02-19T20:39:34.969 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:39:34.968+0000 7f5c11149640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69732) UID: 1000 2026-02-19T20:39:35.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4418573312 bytes 2026-02-19T20:39:35.085 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4484187136 bytes 2026-02-19T20:39:35.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4544392192 bytes 2026-02-19T20:39:35.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4604589056 bytes 2026-02-19T20:39:35.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4648663040 bytes 2026-02-19T20:39:35.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4671245312 bytes 2026-02-19T20:39:35.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4719673344 bytes 2026-02-19T20:39:35.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4729661440 bytes 2026-02-19T20:39:35.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4804876288 bytes 2026-02-19T20:39:35.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4809576448 bytes 2026-02-19T20:39:35.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4868140032 bytes 2026-02-19T20:39:35.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4906446848 bytes 2026-02-19T20:39:35.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4948315136 bytes 2026-02-19T20:39:35.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4960852992 bytes 2026-02-19T20:39:35.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5001227264 bytes 2026-02-19T20:39:35.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5025286144 bytes 2026-02-19T20:39:35.966 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5070326784 bytes 2026-02-19T20:39:36.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5130710016 bytes 2026-02-19T20:39:36.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5190281216 bytes 2026-02-19T20:39:36.167 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5236113408 bytes 2026-02-19T20:39:36.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5240947712 bytes 2026-02-19T20:39:36.255 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5285278720 bytes 2026-02-19T20:39:36.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5315879936 bytes 2026-02-19T20:39:36.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5343423488 bytes 2026-02-19T20:39:36.441 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5444663296 bytes 2026-02-19T20:39:36.525 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5519715328 bytes 2026-02-19T20:39:36.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5520333824 bytes 2026-02-19T20:39:36.626 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5574866944 bytes 2026-02-19T20:39:36.693 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5629911040 bytes 2026-02-19T20:39:36.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5670033408 bytes 2026-02-19T20:39:36.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5685872640 bytes 2026-02-19T20:39:36.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5731397632 bytes 2026-02-19T20:39:36.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5768498176 bytes 2026-02-19T20:39:36.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5816036352 bytes 2026-02-19T20:39:37.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5856970752 bytes 2026-02-19T20:39:37.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5932997632 bytes 2026-02-19T20:39:37.175 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5981860864 bytes 2026-02-19T20:39:37.235 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6033786880 bytes 2026-02-19T20:39:37.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6060389376 bytes 2026-02-19T20:39:37.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6104247296 bytes 2026-02-19T20:39:37.397 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6137216000 bytes 2026-02-19T20:39:37.500 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6227034112 bytes 2026-02-19T20:39:37.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6275622912 bytes 2026-02-19T20:39:37.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6353155072 bytes 2026-02-19T20:39:37.669 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6353756160 bytes 2026-02-19T20:39:37.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6397592576 bytes 2026-02-19T20:39:37.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6446614528 bytes 2026-02-19T20:39:37.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6473539584 bytes 2026-02-19T20:39:37.930 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6547691520 bytes 2026-02-19T20:39:37.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6567313408 bytes 2026-02-19T20:39:38.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6614409216 bytes 2026-02-19T20:39:38.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6633576448 bytes 2026-02-19T20:39:38.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6697053184 bytes 2026-02-19T20:39:38.209 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6745033728 bytes 2026-02-19T20:39:38.291 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6807429120 bytes 2026-02-19T20:39:38.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6847104000 bytes 2026-02-19T20:39:38.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6892305408 bytes 2026-02-19T20:39:38.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6964097024 bytes 2026-02-19T20:39:38.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6968636416 bytes 2026-02-19T20:39:38.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6986456064 bytes 2026-02-19T20:39:38.657 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7051417600 bytes 2026-02-19T20:39:38.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7064285184 bytes 2026-02-19T20:39:38.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7133258752 bytes 2026-02-19T20:39:38.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7203995648 bytes 2026-02-19T20:39:38.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7307542528 bytes 2026-02-19T20:39:39.042 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7393180672 bytes 2026-02-19T20:39:39.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7393782784 bytes 2026-02-19T20:39:39.107 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2026-02-19T20:39:39.107 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:39:39.108 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2026-02-19T20:39:39.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2026-02-19T20:39:39.108 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2026-02-19T20:39:39.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2026-02-19T20:39:39.111 INFO:tasks.rbd_mirror_thrash:waiting for 79.89639872232559 secs before thrashing 2026-02-19T20:39:39.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7462706176 bytes 2026-02-19T20:39:39.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7514810368 bytes 2026-02-19T20:39:39.286 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7562467328 bytes 2026-02-19T20:39:39.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7586746368 bytes 2026-02-19T20:39:39.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7611459584 bytes 2026-02-19T20:39:39.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7648922624 bytes 2026-02-19T20:39:39.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7724492800 bytes 2026-02-19T20:39:39.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7847578624 bytes 2026-02-19T20:39:39.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7866320896 bytes 2026-02-19T20:39:39.725 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7898959872 bytes 2026-02-19T20:39:39.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:39:39.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:39:39.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:39:39.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:39:39.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_6 present 2026-02-19T20:39:39.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:39:39.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:39.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:39.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:39:39.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:39:39.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:39:39.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:39:39.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_6 2026-02-19T20:39:39.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:39:39.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:39:39.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_6 2026-02-19T20:39:39.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2026-02-19T20:39:39.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:39:39.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113abb64be5e 2026-02-19T20:39:39.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:39:39.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:39:39.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_6 present 113abb64be5e 2026-02-19T20:39:39.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:39:39.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:39.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:39.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:39:39.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113abb64be5e 2026-02-19T20:39:39.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:39:39.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:39:39.878 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_6 2026-02-19T20:39:39.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:39:39.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:39:39.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_6 2026-02-19T20:39:39.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2026-02-19T20:39:39.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:39:39.902 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113abb64be5e 2026-02-19T20:39:39.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113abb64be5e 2026-02-19T20:39:39.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113abb64be5e -o 113abb64be5e = 113abb64be5e 2026-02-19T20:39:39.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:39:39.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:39:39.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:39:39.903 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_6 2026-02-19T20:39:39.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:39.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:39.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:39.904 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_6 2026-02-19T20:39:40.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:39:40.644+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x3e69c00 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=190 2026-02-19T20:39:40.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:39:40.644+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x3e69c00 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=190, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=be, last_copied_object_number=128, snap_seqs={190=18446744073709551614}] 2026-02-19T20:39:45.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:39:45.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_6 deleted 2026-02-19T20:39:45.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:39:45.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:45.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:45.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:39:45.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:39:45.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:39:45.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:39:45.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_6 2026-02-19T20:39:45.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:39:45.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:39:45.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_6 2026-02-19T20:39:45.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2026-02-19T20:39:45.656 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:39:45.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_6: (2) No such file or directory 2026-02-19T20:39:45.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:39:45.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:39:45.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:39:45.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_6 deleted '' 2026-02-19T20:39:45.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:39:45.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:45.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:45.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:39:45.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:39:45.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:39:45.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:39:45.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_6 2026-02-19T20:39:45.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:39:45.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:39:45.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_6 2026-02-19T20:39:45.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_6 2026-02-19T20:39:45.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:39:45.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_6: (2) No such file or directory 2026-02-19T20:39:45.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:39:45.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:39:45.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:39:45.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:39:45.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_6 2026-02-19T20:39:45.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:45.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:45.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:45.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_6 2026-02-19T20:39:45.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_6 2026-02-19T20:39:45.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:45.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:45.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:45.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:39:45.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:39:45.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_6 2026-02-19T20:39:45.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:45.833 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:45.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:45.834 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_6 2026-02-19T20:39:45.857 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_6 2026-02-19T20:39:45.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:39:45.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:39:45.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:39:45.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:39:45.884 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:39:46.885 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_6 2026-02-19T20:39:46.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:46.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:46.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:46.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_6 2026-02-19T20:39:46.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_6 2026-02-19T20:39:46.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:39:46.938 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:39:46.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:39:46.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:39:46.940 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:39:48.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_6 2026-02-19T20:39:48.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:48.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:48.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:48.941 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_6 2026-02-19T20:39:48.969 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_6 2026-02-19T20:39:48.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:39:48.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:39:48.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:39:48.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:39:48.994 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:39:52.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_6 2026-02-19T20:39:52.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:39:52.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:39:52.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:39:52.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_6 2026-02-19T20:39:53.024 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_6 2026-02-19T20:39:53.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:39:53.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:39:53.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:39:53.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:39:53.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:40:01.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_6 2026-02-19T20:40:01.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:01.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:01.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:40:01.053 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_6 2026-02-19T20:40:01.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_6 2026-02-19T20:40:01.107 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:40:01.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:40:01.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:40:01.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:40:01.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:40:17.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_6 2026-02-19T20:40:17.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:17.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:17.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_6 2026-02-19T20:40:17.111 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_6 2026-02-19T20:40:17.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_6 2026-02-19T20:40:17.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:40:17.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:40:17.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:40:17.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_7 2026-02-19T20:40:17.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_7 512M 2026-02-19T20:40:17.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:17.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:40:17.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:17.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:40:17.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:17.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:40:17.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:40:17.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:40:17.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:40:17.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:40:17.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_7 2026-02-19T20:40:17.271 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_7 2026-02-19T20:40:17.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:17.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:17.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:17.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:40:17.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_7 snapshot 2026-02-19T20:40:17.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:40:17.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_7 2026-02-19T20:40:17.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_7': 2026-02-19T20:40:17.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:40:17.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:40:17.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:40:17.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1a78c8a31b08 2026-02-19T20:40:17.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1a78c8a31b08 2026-02-19T20:40:17.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:40:17 2026 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:40:17 2026 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:40:17 2026 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 7b91ba28-87eb-4dae-ae56-1ab48497ff8f 2026-02-19T20:40:17.749 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:40:17.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_7 2026-02-19T20:40:17.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:17.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:17.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:17.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:40:17.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=8 2026-02-19T20:40:17.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:40:17.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_7 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:40:17.854 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 61894656 bytes 2026-02-19T20:40:17.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 82736128 bytes 2026-02-19T20:40:17.962 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 126570496 bytes 2026-02-19T20:40:18.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 138918912 bytes 2026-02-19T20:40:18.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 162382848 bytes 2026-02-19T20:40:18.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 252091392 bytes 2026-02-19T20:40:18.193 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 309062656 bytes 2026-02-19T20:40:18.230 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 309691392 bytes 2026-02-19T20:40:18.273 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 340589568 bytes 2026-02-19T20:40:18.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 401909760 bytes 2026-02-19T20:40:18.505 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 581394432 bytes 2026-02-19T20:40:18.573 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 615915520 bytes 2026-02-19T20:40:18.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 654956544 bytes 2026-02-19T20:40:18.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 675245056 bytes 2026-02-19T20:40:18.720 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 709693440 bytes 2026-02-19T20:40:18.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 723462144 bytes 2026-02-19T20:40:18.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 751877120 bytes 2026-02-19T20:40:18.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 801363968 bytes 2026-02-19T20:40:18.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 854291456 bytes 2026-02-19T20:40:19.016 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 888567808 bytes 2026-02-19T20:40:19.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 920012800 bytes 2026-02-19T20:40:19.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 969627648 bytes 2026-02-19T20:40:19.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1022038016 bytes 2026-02-19T20:40:19.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1077884928 bytes 2026-02-19T20:40:19.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1120571392 bytes 2026-02-19T20:40:19.402 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1188275200 bytes 2026-02-19T20:40:19.478 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1250308096 bytes 2026-02-19T20:40:19.559 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1298434048 bytes 2026-02-19T20:40:19.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1380653056 bytes 2026-02-19T20:40:19.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1426932736 bytes 2026-02-19T20:40:19.786 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1495653376 bytes 2026-02-19T20:40:20.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1527311360 bytes 2026-02-19T20:40:20.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1547261952 bytes 2026-02-19T20:40:20.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1631784960 bytes 2026-02-19T20:40:20.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1659837440 bytes 2026-02-19T20:40:20.283 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1704311808 bytes 2026-02-19T20:40:20.365 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1781299200 bytes 2026-02-19T20:40:20.464 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1853084672 bytes 2026-02-19T20:40:20.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1889136640 bytes 2026-02-19T20:40:20.566 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1909443584 bytes 2026-02-19T20:40:20.647 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1993105408 bytes 2026-02-19T20:40:20.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 2039272448 bytes 2026-02-19T20:40:20.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2104271872 bytes 2026-02-19T20:40:20.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2154358784 bytes 2026-02-19T20:40:20.929 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2190185472 bytes 2026-02-19T20:40:20.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2234484736 bytes 2026-02-19T20:40:21.050 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2274944000 bytes 2026-02-19T20:40:21.100 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2304928768 bytes 2026-02-19T20:40:21.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2345117696 bytes 2026-02-19T20:40:21.229 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2411325440 bytes 2026-02-19T20:40:21.281 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2442083328 bytes 2026-02-19T20:40:21.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2490518528 bytes 2026-02-19T20:40:21.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2547591168 bytes 2026-02-19T20:40:21.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2618736640 bytes 2026-02-19T20:40:21.552 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2662588416 bytes 2026-02-19T20:40:21.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2724670464 bytes 2026-02-19T20:40:21.689 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2788854784 bytes 2026-02-19T20:40:21.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2794207232 bytes 2026-02-19T20:40:21.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2798179328 bytes 2026-02-19T20:40:21.798 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2805262336 bytes 2026-02-19T20:40:21.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2927204352 bytes 2026-02-19T20:40:21.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2939897856 bytes 2026-02-19T20:40:21.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2960242688 bytes 2026-02-19T20:40:22.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2991620096 bytes 2026-02-19T20:40:22.108 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 3030991872 bytes 2026-02-19T20:40:22.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 3100181504 bytes 2026-02-19T20:40:22.243 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 3130470400 bytes 2026-02-19T20:40:22.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 3173591040 bytes 2026-02-19T20:40:22.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 3205647360 bytes 2026-02-19T20:40:22.390 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3240260608 bytes 2026-02-19T20:40:22.459 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3272346624 bytes 2026-02-19T20:40:22.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3304301568 bytes 2026-02-19T20:40:22.547 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3313049600 bytes 2026-02-19T20:40:22.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3377455104 bytes 2026-02-19T20:40:22.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3436704768 bytes 2026-02-19T20:40:22.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3441326080 bytes 2026-02-19T20:40:22.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3471563776 bytes 2026-02-19T20:40:22.810 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3501831168 bytes 2026-02-19T20:40:22.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3605364736 bytes 2026-02-19T20:40:22.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3631106048 bytes 2026-02-19T20:40:23.014 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3661410304 bytes 2026-02-19T20:40:23.098 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3723142144 bytes 2026-02-19T20:40:23.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3785895936 bytes 2026-02-19T20:40:23.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3802515456 bytes 2026-02-19T20:40:23.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3803113472 bytes 2026-02-19T20:40:23.272 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3810574336 bytes 2026-02-19T20:40:23.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3836040192 bytes 2026-02-19T20:40:23.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3874625536 bytes 2026-02-19T20:40:23.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3944768512 bytes 2026-02-19T20:40:23.528 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3979253760 bytes 2026-02-19T20:40:23.624 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4093246464 bytes 2026-02-19T20:40:23.660 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4093873152 bytes 2026-02-19T20:40:23.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4142132224 bytes 2026-02-19T20:40:23.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4198256640 bytes 2026-02-19T20:40:23.839 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4241054720 bytes 2026-02-19T20:40:23.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4251636736 bytes 2026-02-19T20:40:23.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4323873792 bytes 2026-02-19T20:40:24.025 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4366366720 bytes 2026-02-19T20:40:24.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4396530688 bytes 2026-02-19T20:40:24.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4473570304 bytes 2026-02-19T20:40:24.195 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4474184704 bytes 2026-02-19T20:40:24.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4529579008 bytes 2026-02-19T20:40:24.301 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4551731200 bytes 2026-02-19T20:40:24.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4576520192 bytes 2026-02-19T20:40:24.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4579742720 bytes 2026-02-19T20:40:24.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4603077632 bytes 2026-02-19T20:40:24.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4641321984 bytes 2026-02-19T20:40:24.524 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4648954880 bytes 2026-02-19T20:40:24.598 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4716247040 bytes 2026-02-19T20:40:24.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4755077120 bytes 2026-02-19T20:40:24.712 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4795942912 bytes 2026-02-19T20:40:24.751 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4818395136 bytes 2026-02-19T20:40:24.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4853672960 bytes 2026-02-19T20:40:24.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4860605440 bytes 2026-02-19T20:40:24.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4885259264 bytes 2026-02-19T20:40:24.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4923416576 bytes 2026-02-19T20:40:25.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4951077888 bytes 2026-02-19T20:40:25.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4988096512 bytes 2026-02-19T20:40:25.112 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5022633984 bytes 2026-02-19T20:40:25.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5097740288 bytes 2026-02-19T20:40:25.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5123233792 bytes 2026-02-19T20:40:25.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5183523840 bytes 2026-02-19T20:40:25.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5228734464 bytes 2026-02-19T20:40:25.437 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5295592448 bytes 2026-02-19T20:40:25.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5336945664 bytes 2026-02-19T20:40:25.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5397909504 bytes 2026-02-19T20:40:25.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5463364608 bytes 2026-02-19T20:40:25.726 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5536438272 bytes 2026-02-19T20:40:25.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_7 present 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_7 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_7 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2026-02-19T20:40:25.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:40:25.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113ac990d0f3 2026-02-19T20:40:25.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:40:25.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:40:25.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_7 present 113ac990d0f3 2026-02-19T20:40:25.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113ac990d0f3 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_7 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_7 2026-02-19T20:40:25.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2026-02-19T20:40:25.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:40:25.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113ac990d0f3 2026-02-19T20:40:25.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113ac990d0f3 2026-02-19T20:40:25.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113ac990d0f3 -o 113ac990d0f3 = 113ac990d0f3 2026-02-19T20:40:25.911 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:40:25.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:40:25.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:40:25.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_7 2026-02-19T20:40:25.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:25.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:25.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:25.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_7 2026-02-19T20:40:26.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:40:25.998+0000 7f43886f4640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5112c00 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=191 2026-02-19T20:40:26.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:40:25.998+0000 7f43886f4640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5112c00 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=191, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=bf, last_copied_object_number=128, snap_seqs={191=18446744073709551614}] 2026-02-19T20:40:26.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:40:26.690+0000 7f5fd0aeb640 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x4e4dba0 handle_prepare_local_image: error preparing local image for replay: (85) Interrupted system call should be restarted 2026-02-19T20:40:26.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:40:26.690+0000 7f5fd72f8640 -1 rbd::mirror::ImageReplayer: 0x4a22000 [3/7b91ba28-87eb-4dae-ae56-1ab48497ff8f] operator(): start failed: (85) Interrupted system call should be restarted 2026-02-19T20:40:31.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:40:31.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_7 deleted 2026-02-19T20:40:31.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:40:31.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:31.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:31.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:40:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:40:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:40:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:40:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_7 2026-02-19T20:40:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:40:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:40:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_7 2026-02-19T20:40:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2026-02-19T20:40:31.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:40:31.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_7: (2) No such file or directory 2026-02-19T20:40:31.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:40:31.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:40:31.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:40:31.136 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_7 deleted '' 2026-02-19T20:40:31.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:40:31.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:31.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:31.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:40:31.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:40:31.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:40:31.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:40:31.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_7 2026-02-19T20:40:31.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:40:31.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:40:31.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_7 2026-02-19T20:40:31.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_7 2026-02-19T20:40:31.138 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:40:31.157 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_7: (2) No such file or directory 2026-02-19T20:40:31.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:40:31.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:40:31.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:40:31.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:40:31.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_7 2026-02-19T20:40:31.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:31.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:31.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:31.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_7 2026-02-19T20:40:31.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_7 2026-02-19T20:40:31.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:31.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:31.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:31.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:40:31.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:40:31.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_7 2026-02-19T20:40:31.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:31.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:31.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:31.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_7 2026-02-19T20:40:31.214 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_7 2026-02-19T20:40:31.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:40:31.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:40:31.240 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:40:31.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:40:31.241 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:40:32.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_7 2026-02-19T20:40:32.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:32.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:32.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:32.242 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_7 2026-02-19T20:40:32.269 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_7 2026-02-19T20:40:32.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:40:32.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:40:32.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:40:32.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:40:32.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:40:34.299 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_7 2026-02-19T20:40:34.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:34.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:34.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:34.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_7 2026-02-19T20:40:34.327 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_7 2026-02-19T20:40:34.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:40:34.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:40:34.353 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:40:34.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:40:34.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:40:37.688 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2026-02-19T20:40:37.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2026-02-19T20:40:37.689 INFO:tasks.rbd_mirror_thrash:waiting for 3.1526462368308215 secs before reviving daemons 2026-02-19T20:40:37.689 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:40:37.687+0000 7ff4fd11e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 76591) UID: 1000 2026-02-19T20:40:38.355 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_7 2026-02-19T20:40:38.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:38.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:38.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:38.356 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_7 2026-02-19T20:40:38.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_7 2026-02-19T20:40:38.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:40:38.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:40:38.412 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:40:38.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:40:38.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:40:40.842 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2026-02-19T20:40:40.843 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:40:40.843 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2026-02-19T20:40:40.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2026-02-19T20:40:40.843 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2026-02-19T20:40:40.846 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2026-02-19T20:40:40.846 INFO:tasks.rbd_mirror_thrash:waiting for 90.29703749143995 secs before thrashing 2026-02-19T20:40:46.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_7 2026-02-19T20:40:46.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:40:46.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:40:46.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:40:46.415 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_7 2026-02-19T20:40:46.444 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_7 2026-02-19T20:40:46.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:40:46.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:40:46.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:40:46.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:40:46.470 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:40:59.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7817580832893295) > weight (0.25) 2026-02-19T20:40:59.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4384368078023796) > weight (0.25) 2026-02-19T20:40:59.008 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2026-02-19T20:40:59.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2026-02-19T20:40:59.009 INFO:tasks.rbd_mirror_thrash:waiting for 8.398523139752303 secs before reviving daemons 2026-02-19T20:40:59.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:40:59.007+0000 7f83e8e4d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 60504) UID: 1000 2026-02-19T20:41:02.471 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_7 2026-02-19T20:41:02.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:02.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:02.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_7 2026-02-19T20:41:02.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_7 2026-02-19T20:41:02.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_7 2026-02-19T20:41:02.607 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:41:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:41:02.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_8 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_8 512M 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:41:02.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:41:02.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:41:02.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_8 2026-02-19T20:41:02.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_8 2026-02-19T20:41:02.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:02.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:02.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:02.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:41:02.641 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_8 snapshot 2026-02-19T20:41:02.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:41:02.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_8 2026-02-19T20:41:02.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_8': 2026-02-19T20:41:02.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:41:02.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:41:02.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:41:02.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1ab7d24722d6 2026-02-19T20:41:02.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1ab7d24722d6 2026-02-19T20:41:02.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:41:02.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:41:02.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:41:02.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:41:02.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:41:02 2026 2026-02-19T20:41:02.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:41:02 2026 2026-02-19T20:41:02.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:41:02 2026 2026-02-19T20:41:02.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:41:02.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:41:02.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: eaf2963f-4b67-4a81-9ce5-2c9ac80f6da4 2026-02-19T20:41:02.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:41:02.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_8 2026-02-19T20:41:02.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:02.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:02.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:02.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:41:02.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=14 2026-02-19T20:41:02.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:41:02.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_8 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:41:02.816 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 611328 bytes 2026-02-19T20:41:02.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 10243072 bytes 2026-02-19T20:41:02.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 72037376 bytes 2026-02-19T20:41:02.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 100646912 bytes 2026-02-19T20:41:03.039 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 165009408 bytes 2026-02-19T20:41:03.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 232046592 bytes 2026-02-19T20:41:03.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 266884096 bytes 2026-02-19T20:41:03.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 286145536 bytes 2026-02-19T20:41:03.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 300428288 bytes 2026-02-19T20:41:03.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 346952704 bytes 2026-02-19T20:41:03.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 380366848 bytes 2026-02-19T20:41:03.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 435883008 bytes 2026-02-19T20:41:03.534 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 561683456 bytes 2026-02-19T20:41:03.585 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 602910720 bytes 2026-02-19T20:41:03.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 678431744 bytes 2026-02-19T20:41:03.814 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 798088192 bytes 2026-02-19T20:41:03.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 863538176 bytes 2026-02-19T20:41:03.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 921153536 bytes 2026-02-19T20:41:04.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 965765120 bytes 2026-02-19T20:41:04.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 983265280 bytes 2026-02-19T20:41:04.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 1022703616 bytes 2026-02-19T20:41:04.222 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 1106646016 bytes 2026-02-19T20:41:04.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1156934656 bytes 2026-02-19T20:41:04.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1204099072 bytes 2026-02-19T20:41:04.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1214912512 bytes 2026-02-19T20:41:04.461 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1294039040 bytes 2026-02-19T20:41:04.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1331938304 bytes 2026-02-19T20:41:04.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1332527104 bytes 2026-02-19T20:41:04.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1345102848 bytes 2026-02-19T20:41:04.696 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1436626944 bytes 2026-02-19T20:41:04.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1494804480 bytes 2026-02-19T20:41:04.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1617057792 bytes 2026-02-19T20:41:04.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1698947072 bytes 2026-02-19T20:41:05.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1752165376 bytes 2026-02-19T20:41:05.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1777737728 bytes 2026-02-19T20:41:05.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1796935680 bytes 2026-02-19T20:41:05.170 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1823220736 bytes 2026-02-19T20:41:05.237 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1873270784 bytes 2026-02-19T20:41:05.321 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1955582976 bytes 2026-02-19T20:41:05.361 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1972234240 bytes 2026-02-19T20:41:05.421 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 2022965248 bytes 2026-02-19T20:41:05.502 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 2073000960 bytes 2026-02-19T20:41:05.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 2093860864 bytes 2026-02-19T20:41:05.577 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2095886336 bytes 2026-02-19T20:41:05.617 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2107245568 bytes 2026-02-19T20:41:05.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2158878720 bytes 2026-02-19T20:41:05.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2241949696 bytes 2026-02-19T20:41:05.799 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2252705792 bytes 2026-02-19T20:41:05.830 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2253338624 bytes 2026-02-19T20:41:05.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2317867008 bytes 2026-02-19T20:41:05.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2365867008 bytes 2026-02-19T20:41:05.995 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2376097792 bytes 2026-02-19T20:41:06.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2388556800 bytes 2026-02-19T20:41:06.092 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2431365120 bytes 2026-02-19T20:41:06.159 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2475609088 bytes 2026-02-19T20:41:06.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2498830336 bytes 2026-02-19T20:41:06.278 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2552974336 bytes 2026-02-19T20:41:06.305 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2553601024 bytes 2026-02-19T20:41:06.333 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2554236928 bytes 2026-02-19T20:41:06.384 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2581520384 bytes 2026-02-19T20:41:06.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2646174720 bytes 2026-02-19T20:41:06.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2734259200 bytes 2026-02-19T20:41:06.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2755342336 bytes 2026-02-19T20:41:06.640 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2764727296 bytes 2026-02-19T20:41:06.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2806945792 bytes 2026-02-19T20:41:06.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2821447680 bytes 2026-02-19T20:41:06.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2843348992 bytes 2026-02-19T20:41:06.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2878383104 bytes 2026-02-19T20:41:06.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2916313088 bytes 2026-02-19T20:41:06.937 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 2916942848 bytes 2026-02-19T20:41:07.000 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 2966874112 bytes 2026-02-19T20:41:07.052 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 2989899776 bytes 2026-02-19T20:41:07.105 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3022048256 bytes 2026-02-19T20:41:07.155 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3052286976 bytes 2026-02-19T20:41:07.219 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3095986176 bytes 2026-02-19T20:41:07.267 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3125764096 bytes 2026-02-19T20:41:07.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3201734656 bytes 2026-02-19T20:41:07.408 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2026-02-19T20:41:07.409 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:41:07.409 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2026-02-19T20:41:07.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2026-02-19T20:41:07.409 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2026-02-19T20:41:07.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2026-02-19T20:41:07.413 INFO:tasks.rbd_mirror_thrash:waiting for 102.98330416277925 secs before thrashing 2026-02-19T20:41:07.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3258685440 bytes 2026-02-19T20:41:07.486 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3290603520 bytes 2026-02-19T20:41:07.576 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3371657216 bytes 2026-02-19T20:41:07.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3402630144 bytes 2026-02-19T20:41:07.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3430236160 bytes 2026-02-19T20:41:07.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3461094400 bytes 2026-02-19T20:41:07.773 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3488177152 bytes 2026-02-19T20:41:07.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3488801792 bytes 2026-02-19T20:41:07.863 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3538823168 bytes 2026-02-19T20:41:07.918 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3553446912 bytes 2026-02-19T20:41:07.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3607821312 bytes 2026-02-19T20:41:08.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3618278400 bytes 2026-02-19T20:41:08.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3735363584 bytes 2026-02-19T20:41:08.207 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 3789243392 bytes 2026-02-19T20:41:08.285 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 3837564928 bytes 2026-02-19T20:41:08.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 3838185472 bytes 2026-02-19T20:41:08.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 3849822208 bytes 2026-02-19T20:41:08.431 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 3942764544 bytes 2026-02-19T20:41:08.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4012094464 bytes 2026-02-19T20:41:08.546 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4021171200 bytes 2026-02-19T20:41:08.625 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4075339776 bytes 2026-02-19T20:41:08.690 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4121790464 bytes 2026-02-19T20:41:08.733 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4159096832 bytes 2026-02-19T20:41:08.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4174135296 bytes 2026-02-19T20:41:08.876 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4249759744 bytes 2026-02-19T20:41:08.908 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4264316928 bytes 2026-02-19T20:41:08.960 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4284386304 bytes 2026-02-19T20:41:08.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4286804992 bytes 2026-02-19T20:41:09.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4314678272 bytes 2026-02-19T20:41:09.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4378998784 bytes 2026-02-19T20:41:09.185 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4474056704 bytes 2026-02-19T20:41:09.249 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4491065344 bytes 2026-02-19T20:41:09.296 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4510164992 bytes 2026-02-19T20:41:09.332 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4524899328 bytes 2026-02-19T20:41:09.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4623558656 bytes 2026-02-19T20:41:09.454 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4624757760 bytes 2026-02-19T20:41:09.521 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4677040128 bytes 2026-02-19T20:41:09.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4700698624 bytes 2026-02-19T20:41:09.645 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4771382272 bytes 2026-02-19T20:41:09.707 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4794519552 bytes 2026-02-19T20:41:09.777 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 4862479360 bytes 2026-02-19T20:41:09.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 4877113344 bytes 2026-02-19T20:41:09.865 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 4901691392 bytes 2026-02-19T20:41:09.906 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 4922595328 bytes 2026-02-19T20:41:09.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 4950246400 bytes 2026-02-19T20:41:09.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 4982103040 bytes 2026-02-19T20:41:10.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5023521792 bytes 2026-02-19T20:41:10.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5132938240 bytes 2026-02-19T20:41:10.215 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5154921472 bytes 2026-02-19T20:41:10.265 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5174816768 bytes 2026-02-19T20:41:10.326 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5221307392 bytes 2026-02-19T20:41:10.388 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5269431296 bytes 2026-02-19T20:41:10.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5300703232 bytes 2026-02-19T20:41:10.508 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5356008448 bytes 2026-02-19T20:41:10.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5395075072 bytes 2026-02-19T20:41:10.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5416203264 bytes 2026-02-19T20:41:10.670 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5461331968 bytes 2026-02-19T20:41:10.719 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5481092096 bytes 2026-02-19T20:41:10.788 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5532201984 bytes 2026-02-19T20:41:10.893 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5635482624 bytes 2026-02-19T20:41:10.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5636069376 bytes 2026-02-19T20:41:10.992 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5686867968 bytes 2026-02-19T20:41:11.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 5751446528 bytes 2026-02-19T20:41:11.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 5802655744 bytes 2026-02-19T20:41:11.203 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 5849961472 bytes 2026-02-19T20:41:11.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 5886604288 bytes 2026-02-19T20:41:11.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 5924658176 bytes 2026-02-19T20:41:11.359 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 5948546048 bytes 2026-02-19T20:41:11.403 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 5974685696 bytes 2026-02-19T20:41:11.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6059972608 bytes 2026-02-19T20:41:11.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6105042944 bytes 2026-02-19T20:41:11.610 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6150350848 bytes 2026-02-19T20:41:11.685 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6197671936 bytes 2026-02-19T20:41:11.734 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6220674048 bytes 2026-02-19T20:41:11.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6251980800 bytes 2026-02-19T20:41:11.820 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6252576768 bytes 2026-02-19T20:41:11.874 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6289528832 bytes 2026-02-19T20:41:11.963 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6390617088 bytes 2026-02-19T20:41:12.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6406805504 bytes 2026-02-19T20:41:12.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6442527744 bytes 2026-02-19T20:41:12.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6503839744 bytes 2026-02-19T20:41:12.162 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6506564608 bytes 2026-02-19T20:41:12.223 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6550285312 bytes 2026-02-19T20:41:12.297 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6607006720 bytes 2026-02-19T20:41:12.345 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6623244288 bytes 2026-02-19T20:41:12.445 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6730430464 bytes 2026-02-19T20:41:12.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6769824768 bytes 2026-02-19T20:41:12.554 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 6799001600 bytes 2026-02-19T20:41:12.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 6834274304 bytes 2026-02-19T20:41:12.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 6867957760 bytes 2026-02-19T20:41:12.727 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 6929570816 bytes 2026-02-19T20:41:12.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 6960559104 bytes 2026-02-19T20:41:12.813 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 6980112384 bytes 2026-02-19T20:41:12.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 6980746240 bytes 2026-02-19T20:41:12.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7054851072 bytes 2026-02-19T20:41:12.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7093428224 bytes 2026-02-19T20:41:13.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7175601152 bytes 2026-02-19T20:41:13.144 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7230507008 bytes 2026-02-19T20:41:13.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7243730944 bytes 2026-02-19T20:41:13.280 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7336803328 bytes 2026-02-19T20:41:13.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7420067840 bytes 2026-02-19T20:41:13.443 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7478402048 bytes 2026-02-19T20:41:13.507 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7527204864 bytes 2026-02-19T20:41:13.584 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7580823552 bytes 2026-02-19T20:41:13.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7628945408 bytes 2026-02-19T20:41:13.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7669919744 bytes 2026-02-19T20:41:13.775 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 7727996928 bytes 2026-02-19T20:41:13.818 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 7734517760 bytes 2026-02-19T20:41:13.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 7748897792 bytes 2026-02-19T20:41:13.922 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 7778169856 bytes 2026-02-19T20:41:13.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 7809616896 bytes 2026-02-19T20:41:14.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 7877803008 bytes 2026-02-19T20:41:14.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 7920605184 bytes 2026-02-19T20:41:14.158 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 7941155840 bytes 2026-02-19T20:41:14.216 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 7998100480 bytes 2026-02-19T20:41:14.258 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 7998711808 bytes 2026-02-19T20:41:14.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8033497088 bytes 2026-02-19T20:41:14.364 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8061405184 bytes 2026-02-19T20:41:14.405 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8083009536 bytes 2026-02-19T20:41:14.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8104081408 bytes 2026-02-19T20:41:14.482 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8104681472 bytes 2026-02-19T20:41:14.512 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8105309184 bytes 2026-02-19T20:41:14.580 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8169614336 bytes 2026-02-19T20:41:14.649 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8203720704 bytes 2026-02-19T20:41:14.703 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 8242470912 bytes 2026-02-19T20:41:14.795 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 8340125696 bytes 2026-02-19T20:41:14.853 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 8390199296 bytes 2026-02-19T20:41:14.927 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 8440741888 bytes 2026-02-19T20:41:14.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 8459873280 bytes 2026-02-19T20:41:15.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20700 IOs, 8534134784 bytes 2026-02-19T20:41:15.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20800 IOs, 8584449024 bytes 2026-02-19T20:41:15.187 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20900 IOs, 8626039808 bytes 2026-02-19T20:41:15.245 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21000 IOs, 8671504384 bytes 2026-02-19T20:41:15.307 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21100 IOs, 8723627008 bytes 2026-02-19T20:41:15.344 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21200 IOs, 8739661824 bytes 2026-02-19T20:41:15.386 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21300 IOs, 8758056960 bytes 2026-02-19T20:41:15.428 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21400 IOs, 8774919168 bytes 2026-02-19T20:41:15.458 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21500 IOs, 8775516160 bytes 2026-02-19T20:41:15.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21600 IOs, 8850042880 bytes 2026-02-19T20:41:15.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21700 IOs, 8929349632 bytes 2026-02-19T20:41:15.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21800 IOs, 8996836352 bytes 2026-02-19T20:41:15.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:21900 IOs, 9014839296 bytes 2026-02-19T20:41:15.797 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22000 IOs, 9035006976 bytes 2026-02-19T20:41:15.910 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22100 IOs, 9160358912 bytes 2026-02-19T20:41:15.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22200 IOs, 9245243392 bytes 2026-02-19T20:41:16.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22300 IOs, 9293492224 bytes 2026-02-19T20:41:16.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22400 IOs, 9310924800 bytes 2026-02-19T20:41:16.169 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22500 IOs, 9341666304 bytes 2026-02-19T20:41:16.224 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22600 IOs, 9383778304 bytes 2026-02-19T20:41:16.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22700 IOs, 9424921600 bytes 2026-02-19T20:41:16.322 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22800 IOs, 9438598144 bytes 2026-02-19T20:41:16.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:22900 IOs, 9514814464 bytes 2026-02-19T20:41:16.489 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23000 IOs, 9561733120 bytes 2026-02-19T20:41:16.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23100 IOs, 9586587648 bytes 2026-02-19T20:41:16.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23200 IOs, 9591895040 bytes 2026-02-19T20:41:16.612 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23300 IOs, 9609647104 bytes 2026-02-19T20:41:16.672 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23400 IOs, 9643607040 bytes 2026-02-19T20:41:16.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:23500 IOs, 9713267712 bytes 2026-02-19T20:41:16.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:41:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:41:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:41:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:41:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_8 present 2026-02-19T20:41:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:41:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:41:16.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:41:16.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:41:16.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:41:16.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_8 2026-02-19T20:41:16.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:41:16.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:41:16.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_8 2026-02-19T20:41:16.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2026-02-19T20:41:16.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:41:16.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113a1ec30d91 2026-02-19T20:41:16.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:41:16.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:41:16.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_8 present 113a1ec30d91 2026-02-19T20:41:16.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:41:16.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:16.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:16.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:41:16.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113a1ec30d91 2026-02-19T20:41:16.894 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:41:16.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:41:16.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_8 2026-02-19T20:41:16.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:41:16.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:41:16.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_8 2026-02-19T20:41:16.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2026-02-19T20:41:16.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113a1ec30d91 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113a1ec30d91 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113a1ec30d91 -o 113a1ec30d91 = 113a1ec30d91 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_8 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:16.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_8 2026-02-19T20:41:17.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:41:17.712+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5112400 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=192 2026-02-19T20:41:17.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:41:17.712+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5112400 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=192, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=c0, last_copied_object_number=128, snap_seqs={192=18446744073709551614}] 2026-02-19T20:41:22.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:41:22.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_8 deleted 2026-02-19T20:41:22.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:41:22.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:22.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:22.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:41:22.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:41:22.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:41:22.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:41:22.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_8 2026-02-19T20:41:22.723 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:41:22.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:41:22.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_8 2026-02-19T20:41:22.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2026-02-19T20:41:22.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:41:22.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_8: (2) No such file or directory 2026-02-19T20:41:22.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:41:22.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:41:22.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:41:22.847 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_8 deleted '' 2026-02-19T20:41:22.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:41:22.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:22.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:22.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:41:22.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:41:22.848 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:41:22.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:41:22.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_8 2026-02-19T20:41:22.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:41:22.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:41:22.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_8 2026-02-19T20:41:22.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_8 2026-02-19T20:41:22.849 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:41:22.867 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_8: (2) No such file or directory 2026-02-19T20:41:22.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:41:22.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:41:22.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:41:22.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:41:22.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_8 2026-02-19T20:41:22.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:22.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:22.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:22.870 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_8 2026-02-19T20:41:22.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_8 2026-02-19T20:41:22.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:22.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:22.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:22.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:41:22.899 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:41:22.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_8 2026-02-19T20:41:22.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:22.900 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:22.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:22.901 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_8 2026-02-19T20:41:22.926 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_8 2026-02-19T20:41:22.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:41:22.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:41:22.951 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:41:22.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:41:22.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:41:23.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_8 2026-02-19T20:41:23.953 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:23.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:23.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:23.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_8 2026-02-19T20:41:23.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_8 2026-02-19T20:41:24.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:41:24.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:41:24.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:41:24.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:41:24.008 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:41:26.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_8 2026-02-19T20:41:26.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:26.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:26.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:26.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_8 2026-02-19T20:41:26.036 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_8 2026-02-19T20:41:26.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:41:26.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:41:26.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:41:26.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:41:26.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:41:30.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_8 2026-02-19T20:41:30.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:30.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:30.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:30.064 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_8 2026-02-19T20:41:30.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_8 2026-02-19T20:41:30.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:41:30.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:41:30.118 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:41:30.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:41:30.120 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:41:38.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_8 2026-02-19T20:41:38.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:38.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:38.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:38.122 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_8 2026-02-19T20:41:38.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_8 2026-02-19T20:41:38.177 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:41:38.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:41:38.178 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:41:38.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:41:38.179 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:41:54.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_8 2026-02-19T20:41:54.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:54.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:54.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_8 2026-02-19T20:41:54.180 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_8 2026-02-19T20:41:54.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_8 2026-02-19T20:41:54.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_9 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_9 512M 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:41:54.517 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_9 2026-02-19T20:41:54.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_9 2026-02-19T20:41:54.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:54.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:54.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:41:54.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:41:54.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_9 snapshot 2026-02-19T20:41:54.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:41:54.732 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_9 2026-02-19T20:41:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_9': 2026-02-19T20:41:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:41:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:41:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:41:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1aeec1fb987d 2026-02-19T20:41:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1aeec1fb987d 2026-02-19T20:41:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:41:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:41:54.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:41:54.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:41:54.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:41:54 2026 2026-02-19T20:41:54.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:41:54 2026 2026-02-19T20:41:54.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:41:54 2026 2026-02-19T20:41:54.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:41:54.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:41:54.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 5f3c5e22-f708-4e37-864a-e876b505d43a 2026-02-19T20:41:54.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:41:54.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_9 2026-02-19T20:41:54.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:41:54.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:41:54.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:41:54.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:41:54.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=13 2026-02-19T20:41:54.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:41:54.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_9 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:41:55.038 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 49849344 bytes 2026-02-19T20:41:55.127 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 131503104 bytes 2026-02-19T20:41:55.173 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 151258112 bytes 2026-02-19T20:41:55.204 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 157096960 bytes 2026-02-19T20:41:55.254 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 194201600 bytes 2026-02-19T20:41:55.293 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 218043392 bytes 2026-02-19T20:41:55.357 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 263898112 bytes 2026-02-19T20:41:55.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 319990784 bytes 2026-02-19T20:41:55.476 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 355084288 bytes 2026-02-19T20:41:55.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 432548864 bytes 2026-02-19T20:41:55.637 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 515737600 bytes 2026-02-19T20:41:55.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 553460736 bytes 2026-02-19T20:41:55.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 615932928 bytes 2026-02-19T20:41:55.851 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 671528960 bytes 2026-02-19T20:41:55.887 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 687144960 bytes 2026-02-19T20:41:55.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 741421056 bytes 2026-02-19T20:41:56.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 764243968 bytes 2026-02-19T20:41:56.073 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 847001600 bytes 2026-02-19T20:41:56.147 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 889913344 bytes 2026-02-19T20:41:56.205 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 938832896 bytes 2026-02-19T20:41:56.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 977313792 bytes 2026-02-19T20:41:56.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 994539520 bytes 2026-02-19T20:41:56.382 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 1073217536 bytes 2026-02-19T20:41:56.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 1110645760 bytes 2026-02-19T20:41:56.488 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 1145945088 bytes 2026-02-19T20:41:56.545 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1184823296 bytes 2026-02-19T20:41:56.593 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1212380160 bytes 2026-02-19T20:41:56.644 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1245282304 bytes 2026-02-19T20:41:56.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1267129344 bytes 2026-02-19T20:41:56.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1323956224 bytes 2026-02-19T20:41:56.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1382630400 bytes 2026-02-19T20:41:56.886 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1421590528 bytes 2026-02-19T20:41:56.949 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1461336064 bytes 2026-02-19T20:41:57.012 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1502583808 bytes 2026-02-19T20:41:57.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1539891200 bytes 2026-02-19T20:41:57.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1565897728 bytes 2026-02-19T20:41:57.168 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1592741888 bytes 2026-02-19T20:41:57.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1674137600 bytes 2026-02-19T20:41:57.314 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1740175360 bytes 2026-02-19T20:41:57.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1788322816 bytes 2026-02-19T20:41:57.439 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1815817216 bytes 2026-02-19T20:41:57.495 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1862668288 bytes 2026-02-19T20:41:57.549 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1879312384 bytes 2026-02-19T20:41:57.632 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 1956672512 bytes 2026-02-19T20:41:57.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 1979901952 bytes 2026-02-19T20:41:57.722 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 1992111104 bytes 2026-02-19T20:41:57.785 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2050590720 bytes 2026-02-19T20:41:57.896 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2156600320 bytes 2026-02-19T20:41:57.934 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2169923584 bytes 2026-02-19T20:41:57.993 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2207619072 bytes 2026-02-19T20:41:58.055 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2262717440 bytes 2026-02-19T20:41:58.114 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2294089728 bytes 2026-02-19T20:41:58.154 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2310861824 bytes 2026-02-19T20:41:58.210 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2352709632 bytes 2026-02-19T20:41:58.289 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2425547776 bytes 2026-02-19T20:41:58.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2467875840 bytes 2026-02-19T20:41:58.436 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2552061952 bytes 2026-02-19T20:41:58.510 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2621094912 bytes 2026-02-19T20:41:58.550 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2635863040 bytes 2026-02-19T20:41:58.603 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2666594304 bytes 2026-02-19T20:41:58.675 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2734428160 bytes 2026-02-19T20:41:58.721 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2761320448 bytes 2026-02-19T20:41:58.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2805201920 bytes 2026-02-19T20:41:58.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2842994688 bytes 2026-02-19T20:41:58.882 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2864482304 bytes 2026-02-19T20:41:58.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2868294656 bytes 2026-02-19T20:41:58.965 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2900269056 bytes 2026-02-19T20:41:59.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2918917120 bytes 2026-02-19T20:41:59.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2982372352 bytes 2026-02-19T20:41:59.137 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3020165120 bytes 2026-02-19T20:41:59.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3056428032 bytes 2026-02-19T20:41:59.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3115425792 bytes 2026-02-19T20:41:59.561 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3249090560 bytes 2026-02-19T20:41:59.609 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3274459136 bytes 2026-02-19T20:41:59.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3354262528 bytes 2026-02-19T20:41:59.742 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3395338240 bytes 2026-02-19T20:41:59.826 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3474111488 bytes 2026-02-19T20:41:59.877 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3511009280 bytes 2026-02-19T20:41:59.921 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3546799104 bytes 2026-02-19T20:41:59.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3575530496 bytes 2026-02-19T20:42:00.018 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3576179712 bytes 2026-02-19T20:42:00.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3610665984 bytes 2026-02-19T20:42:00.152 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3710948352 bytes 2026-02-19T20:42:00.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3733343232 bytes 2026-02-19T20:42:00.447 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3743368192 bytes 2026-02-19T20:42:00.514 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3796848640 bytes 2026-02-19T20:42:00.601 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3879006208 bytes 2026-02-19T20:42:00.651 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3935587328 bytes 2026-02-19T20:42:00.699 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3936195584 bytes 2026-02-19T20:42:00.754 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3972290560 bytes 2026-02-19T20:42:00.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4020178944 bytes 2026-02-19T20:42:00.875 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4078198784 bytes 2026-02-19T20:42:00.955 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4144718848 bytes 2026-02-19T20:42:00.984 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4150977536 bytes 2026-02-19T20:42:01.044 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4186070016 bytes 2026-02-19T20:42:01.109 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4247943168 bytes 2026-02-19T20:42:01.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4348937216 bytes 2026-02-19T20:42:01.324 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4441199616 bytes 2026-02-19T20:42:01.371 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4469475328 bytes 2026-02-19T20:42:01.427 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4505845760 bytes 2026-02-19T20:42:01.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4543468544 bytes 2026-02-19T20:42:01.530 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4568017920 bytes 2026-02-19T20:42:01.572 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4591031296 bytes 2026-02-19T20:42:01.615 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4612570112 bytes 2026-02-19T20:42:01.661 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4640475136 bytes 2026-02-19T20:42:01.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4703328256 bytes 2026-02-19T20:42:01.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4743434240 bytes 2026-02-19T20:42:01.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4777777152 bytes 2026-02-19T20:42:01.909 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4833362944 bytes 2026-02-19T20:42:01.967 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4859941888 bytes 2026-02-19T20:42:02.011 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4892926976 bytes 2026-02-19T20:42:02.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4913427456 bytes 2026-02-19T20:42:02.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4945900544 bytes 2026-02-19T20:42:02.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 5025191936 bytes 2026-02-19T20:42:02.247 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 5047728128 bytes 2026-02-19T20:42:02.313 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 5109797888 bytes 2026-02-19T20:42:02.358 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 5148155904 bytes 2026-02-19T20:42:02.453 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5255193600 bytes 2026-02-19T20:42:02.516 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5278144512 bytes 2026-02-19T20:42:02.578 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5338320896 bytes 2026-02-19T20:42:02.623 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5351183360 bytes 2026-02-19T20:42:02.680 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5391842304 bytes 2026-02-19T20:42:02.736 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5425869824 bytes 2026-02-19T20:42:02.774 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5437033472 bytes 2026-02-19T20:42:02.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5467097088 bytes 2026-02-19T20:42:02.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5539299328 bytes 2026-02-19T20:42:02.981 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5617172480 bytes 2026-02-19T20:42:03.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5672565760 bytes 2026-02-19T20:42:03.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5690233856 bytes 2026-02-19T20:42:03.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5733029888 bytes 2026-02-19T20:42:03.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5771732992 bytes 2026-02-19T20:42:03.275 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5840462848 bytes 2026-02-19T20:42:03.328 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5880326144 bytes 2026-02-19T20:42:03.370 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5881382912 bytes 2026-02-19T20:42:03.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5916909568 bytes 2026-02-19T20:42:03.498 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5972853760 bytes 2026-02-19T20:42:03.533 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5984191488 bytes 2026-02-19T20:42:03.560 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5984811008 bytes 2026-02-19T20:42:03.619 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 6055205888 bytes 2026-02-19T20:42:03.695 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6097500160 bytes 2026-02-19T20:42:03.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6134841344 bytes 2026-02-19T20:42:03.805 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6157882368 bytes 2026-02-19T20:42:03.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6184993792 bytes 2026-02-19T20:42:03.888 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6190638080 bytes 2026-02-19T20:42:03.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6225101824 bytes 2026-02-19T20:42:04.004 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6274816000 bytes 2026-02-19T20:42:04.065 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6318987264 bytes 2026-02-19T20:42:04.104 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6332396544 bytes 2026-02-19T20:42:04.181 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6391299072 bytes 2026-02-19T20:42:04.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6462861312 bytes 2026-02-19T20:42:04.316 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6484739072 bytes 2026-02-19T20:42:04.383 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6538836992 bytes 2026-02-19T20:42:04.420 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6561789952 bytes 2026-02-19T20:42:04.499 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6647574528 bytes 2026-02-19T20:42:04.568 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6696814592 bytes 2026-02-19T20:42:04.597 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6697434112 bytes 2026-02-19T20:42:04.648 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6720713728 bytes 2026-02-19T20:42:04.698 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6740442112 bytes 2026-02-19T20:42:04.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6780897280 bytes 2026-02-19T20:42:04.808 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6806633472 bytes 2026-02-19T20:42:04.879 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6861129728 bytes 2026-02-19T20:42:04.957 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 6944424960 bytes 2026-02-19T20:42:04.997 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 6957124608 bytes 2026-02-19T20:42:05.048 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 6990134272 bytes 2026-02-19T20:42:05.095 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7020141568 bytes 2026-02-19T20:42:05.164 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7083759616 bytes 2026-02-19T20:42:05.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7169982464 bytes 2026-02-19T20:42:05.298 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7202607104 bytes 2026-02-19T20:42:05.369 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7263120384 bytes 2026-02-19T20:42:05.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7298689024 bytes 2026-02-19T20:42:05.501 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7356668928 bytes 2026-02-19T20:42:05.590 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7440232448 bytes 2026-02-19T20:42:05.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7466764288 bytes 2026-02-19T20:42:05.750 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7587215360 bytes 2026-02-19T20:42:05.812 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7631081472 bytes 2026-02-19T20:42:05.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7687572480 bytes 2026-02-19T20:42:05.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7704250368 bytes 2026-02-19T20:42:05.987 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7759060992 bytes 2026-02-19T20:42:06.051 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7815087104 bytes 2026-02-19T20:42:06.119 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7853277184 bytes 2026-02-19T20:42:06.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7973219328 bytes 2026-02-19T20:42:06.261 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 8009232384 bytes 2026-02-19T20:42:06.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 8026937344 bytes 2026-02-19T20:42:06.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 8060037120 bytes 2026-02-19T20:42:06.456 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 8133471232 bytes 2026-02-19T20:42:06.548 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8232036352 bytes 2026-02-19T20:42:06.630 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8336269312 bytes 2026-02-19T20:42:06.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8364201984 bytes 2026-02-19T20:42:06.762 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8429002752 bytes 2026-02-19T20:42:06.832 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8478930944 bytes 2026-02-19T20:42:06.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8537942016 bytes 2026-02-19T20:42:06.954 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8579722240 bytes 2026-02-19T20:42:07.003 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8599334912 bytes 2026-02-19T20:42:07.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8682693632 bytes 2026-02-19T20:42:07.130 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8709700608 bytes 2026-02-19T20:42:07.171 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8736277504 bytes 2026-02-19T20:42:07.246 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19700 IOs, 8799951872 bytes 2026-02-19T20:42:07.325 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19800 IOs, 8864244736 bytes 2026-02-19T20:42:07.385 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19900 IOs, 8920621056 bytes 2026-02-19T20:42:07.433 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20000 IOs, 8945914880 bytes 2026-02-19T20:42:07.497 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20100 IOs, 8986298368 bytes 2026-02-19T20:42:07.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20200 IOs, 9030080512 bytes 2026-02-19T20:42:07.587 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20300 IOs, 9030687744 bytes 2026-02-19T20:42:07.629 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20400 IOs, 9048238080 bytes 2026-02-19T20:42:07.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20500 IOs, 9058132992 bytes 2026-02-19T20:42:07.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:20600 IOs, 9062290432 bytes 2026-02-19T20:42:07.763 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_9 present 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:42:07.764 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:42:07.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:42:07.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_9 2026-02-19T20:42:07.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:42:07.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:42:07.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_9 2026-02-19T20:42:07.765 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2026-02-19T20:42:07.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:42:07.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113afae1429e 2026-02-19T20:42:07.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:42:07.790 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_9 present 113afae1429e 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113afae1429e 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_9 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_9 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2026-02-19T20:42:07.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113afae1429e 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113afae1429e 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113afae1429e -o 113afae1429e = 113afae1429e 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_9 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:07.916 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_9 2026-02-19T20:42:08.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:42:08.732+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5113c00 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=193 2026-02-19T20:42:08.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:42:08.732+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5113c00 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=193, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=c1, last_copied_object_number=128, snap_seqs={193=18446744073709551614}] 2026-02-19T20:42:11.144 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5172440075798114) > weight (0.25) 2026-02-19T20:42:11.144 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8454977885004211) > weight (0.25) 2026-02-19T20:42:11.145 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2026-02-19T20:42:11.145 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2026-02-19T20:42:11.145 INFO:tasks.rbd_mirror_thrash:waiting for 9.108613520557045 secs before reviving daemons 2026-02-19T20:42:11.146 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:42:11.144+0000 7efd7db5c640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 80517) UID: 1000 2026-02-19T20:42:13.738 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:42:13.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_9 deleted 2026-02-19T20:42:13.744 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_9 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_9 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2026-02-19T20:42:13.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:42:13.766 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_9: (2) No such file or directory 2026-02-19T20:42:13.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:42:13.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:42:13.768 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_9 deleted '' 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_9 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_9 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_9 2026-02-19T20:42:13.869 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:42:13.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_9: (2) No such file or directory 2026-02-19T20:42:13.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:42:13.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:42:13.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:42:13.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:42:13.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_9 2026-02-19T20:42:13.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:13.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:13.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:13.892 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_9 2026-02-19T20:42:13.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_9 2026-02-19T20:42:13.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:13.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:13.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:13.919 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:42:13.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:42:13.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_9 2026-02-19T20:42:13.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:13.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:13.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:13.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_9 2026-02-19T20:42:13.945 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_9 2026-02-19T20:42:13.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:42:13.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:42:13.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:42:13.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:42:13.972 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:42:14.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_9 2026-02-19T20:42:14.973 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:14.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:14.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:14.974 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_9 2026-02-19T20:42:15.001 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_9 2026-02-19T20:42:15.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:42:15.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:42:15.027 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:42:15.028 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:42:15.029 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:42:17.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_9 2026-02-19T20:42:17.030 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:17.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:17.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:17.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_9 2026-02-19T20:42:17.057 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_9 2026-02-19T20:42:17.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:42:17.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:42:17.081 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:42:17.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:42:17.082 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:42:20.255 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2026-02-19T20:42:20.255 INFO:teuthology.orchestra.run:waiting for 600 2026-02-19T20:42:20.255 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2026-02-19T20:42:20.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2026-02-19T20:42:20.256 DEBUG:teuthology.orchestra.run.trial080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2026-02-19T20:42:20.259 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2026-02-19T20:42:20.259 INFO:tasks.rbd_mirror_thrash:waiting for 92.10852676235972 secs before thrashing 2026-02-19T20:42:21.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_9 2026-02-19T20:42:21.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:21.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:21.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:21.084 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_9 2026-02-19T20:42:21.113 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_9 2026-02-19T20:42:21.139 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:42:21.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:42:21.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:42:21.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:42:21.141 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:42:29.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_9 2026-02-19T20:42:29.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:29.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:29.142 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:29.143 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_9 2026-02-19T20:42:29.172 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_9 2026-02-19T20:42:29.198 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:42:29.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:42:29.199 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:42:29.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:42:29.200 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:42:45.201 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_9 2026-02-19T20:42:45.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:45.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:45.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_9 2026-02-19T20:42:45.202 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_9 2026-02-19T20:42:45.231 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_9 2026-02-19T20:42:45.335 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:42:45.337 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for i in `seq 1 10` 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image=image_10 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ create_image cluster2 mirror image_10 512M 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local size=128 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -n 512M ']' 2026-02-19T20:42:45.338 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ size=512M 2026-02-19T20:42:45.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ shift 2026-02-19T20:42:45.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering mirror/image_10 2026-02-19T20:42:45.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ enable_mirror cluster2 mirror image_10 2026-02-19T20:42:45.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:45.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:45.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:42:45.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local mode=snapshot 2026-02-19T20:42:45.368 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_10 snapshot 2026-02-19T20:42:45.678 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring enabled 2026-02-19T20:42:45.681 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 info mirror/image_10 2026-02-19T20:42:45.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:rbd image 'image_10': 2026-02-19T20:42:45.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: size 512 MiB in 128 objects 2026-02-19T20:42:45.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: order 22 (4 MiB objects) 2026-02-19T20:42:45.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: snapshot_count: 1 2026-02-19T20:42:45.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: id: 1b2e51736a2e 2026-02-19T20:42:45.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: block_name_prefix: rbd_data.1b2e51736a2e 2026-02-19T20:42:45.704 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: format: 2 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: features: layering 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: op_features: 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: flags: 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: create_timestamp: Thu Feb 19 20:42:45 2026 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: access_timestamp: Thu Feb 19 20:42:45 2026 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: modify_timestamp: Thu Feb 19 20:42:45 2026 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring state: enabled 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring mode: snapshot 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring global id: 08501e84-29c8-4d02-8a21-1b2a7c288cec 2026-02-19T20:42:45.705 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout: mirroring primary: true 2026-02-19T20:42:45.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stress_write_image cluster2 mirror image_10 2026-02-19T20:42:45.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:45.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:45.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:42:45.706 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ awk 'BEGIN {srand(); print int(10 * rand()) + 5}' 2026-02-19T20:42:45.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local duration=12 2026-02-19T20:42:45.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:42:45.708 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_10 --debug-rbd=20 --debug-journaler=20 2026-02-19T20:42:45.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:100 IOs, 34590720 bytes 2026-02-19T20:42:45.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:200 IOs, 75308032 bytes 2026-02-19T20:42:45.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:300 IOs, 136350720 bytes 2026-02-19T20:42:45.952 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:400 IOs, 136970240 bytes 2026-02-19T20:42:46.010 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:500 IOs, 191740928 bytes 2026-02-19T20:42:46.069 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:600 IOs, 218115072 bytes 2026-02-19T20:42:46.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:700 IOs, 240920576 bytes 2026-02-19T20:42:46.189 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:800 IOs, 326478848 bytes 2026-02-19T20:42:46.234 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:900 IOs, 354255872 bytes 2026-02-19T20:42:46.279 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1000 IOs, 377926656 bytes 2026-02-19T20:42:46.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1100 IOs, 417872896 bytes 2026-02-19T20:42:46.372 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1200 IOs, 422684672 bytes 2026-02-19T20:42:46.430 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1300 IOs, 463561728 bytes 2026-02-19T20:42:46.465 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1400 IOs, 483927040 bytes 2026-02-19T20:42:46.519 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1500 IOs, 515907584 bytes 2026-02-19T20:42:46.600 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1600 IOs, 592296960 bytes 2026-02-19T20:42:46.655 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1700 IOs, 631473152 bytes 2026-02-19T20:42:46.686 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1800 IOs, 632083456 bytes 2026-02-19T20:42:46.730 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:1900 IOs, 649586688 bytes 2026-02-19T20:42:46.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2000 IOs, 695445504 bytes 2026-02-19T20:42:46.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2100 IOs, 739460096 bytes 2026-02-19T20:42:46.915 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2200 IOs, 779516928 bytes 2026-02-19T20:42:46.959 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2300 IOs, 801301504 bytes 2026-02-19T20:42:47.045 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2400 IOs, 886311936 bytes 2026-02-19T20:42:47.089 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2500 IOs, 907571200 bytes 2026-02-19T20:42:47.190 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2600 IOs, 1012278272 bytes 2026-02-19T20:42:47.259 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2700 IOs, 1078226944 bytes 2026-02-19T20:42:47.294 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2800 IOs, 1081904128 bytes 2026-02-19T20:42:47.393 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:2900 IOs, 1234964480 bytes 2026-02-19T20:42:47.472 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3000 IOs, 1260364800 bytes 2026-02-19T20:42:47.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3100 IOs, 1263078400 bytes 2026-02-19T20:42:47.543 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3200 IOs, 1274875904 bytes 2026-02-19T20:42:47.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3300 IOs, 1321295872 bytes 2026-02-19T20:42:47.662 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3400 IOs, 1361836032 bytes 2026-02-19T20:42:47.745 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3500 IOs, 1454679040 bytes 2026-02-19T20:42:47.791 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3600 IOs, 1490098176 bytes 2026-02-19T20:42:47.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3700 IOs, 1592557568 bytes 2026-02-19T20:42:47.980 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3800 IOs, 1665705984 bytes 2026-02-19T20:42:48.071 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:3900 IOs, 1744776192 bytes 2026-02-19T20:42:48.125 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4000 IOs, 1780256768 bytes 2026-02-19T20:42:48.213 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4100 IOs, 1878580224 bytes 2026-02-19T20:42:48.253 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4200 IOs, 1900167168 bytes 2026-02-19T20:42:48.311 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4300 IOs, 1955730432 bytes 2026-02-19T20:42:48.413 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4400 IOs, 2041709568 bytes 2026-02-19T20:42:48.462 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4500 IOs, 2065739776 bytes 2026-02-19T20:42:48.529 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4600 IOs, 2118306816 bytes 2026-02-19T20:42:48.563 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4700 IOs, 2122970112 bytes 2026-02-19T20:42:48.618 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4800 IOs, 2162833408 bytes 2026-02-19T20:42:48.676 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:4900 IOs, 2195129344 bytes 2026-02-19T20:42:48.731 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5000 IOs, 2234070016 bytes 2026-02-19T20:42:49.026 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5100 IOs, 2319903744 bytes 2026-02-19T20:42:49.072 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5200 IOs, 2342341632 bytes 2026-02-19T20:42:49.151 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5300 IOs, 2407788544 bytes 2026-02-19T20:42:49.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5400 IOs, 2483951616 bytes 2026-02-19T20:42:49.268 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5500 IOs, 2484560896 bytes 2026-02-19T20:42:49.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5600 IOs, 2500976640 bytes 2026-02-19T20:42:49.343 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5700 IOs, 2505121792 bytes 2026-02-19T20:42:49.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5800 IOs, 2572379136 bytes 2026-02-19T20:42:49.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:5900 IOs, 2649636864 bytes 2026-02-19T20:42:49.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6000 IOs, 2673720320 bytes 2026-02-19T20:42:49.602 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6100 IOs, 2703757312 bytes 2026-02-19T20:42:49.664 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6200 IOs, 2744099840 bytes 2026-02-19T20:42:49.724 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6300 IOs, 2792863744 bytes 2026-02-19T20:42:49.779 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6400 IOs, 2828515328 bytes 2026-02-19T20:42:49.846 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6500 IOs, 2882678784 bytes 2026-02-19T20:42:49.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6600 IOs, 2926558208 bytes 2026-02-19T20:42:49.958 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6700 IOs, 2946408448 bytes 2026-02-19T20:42:50.009 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6800 IOs, 2968437760 bytes 2026-02-19T20:42:50.043 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:6900 IOs, 2974666752 bytes 2026-02-19T20:42:50.106 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7000 IOs, 3025625088 bytes 2026-02-19T20:42:50.135 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7100 IOs, 3037935616 bytes 2026-02-19T20:42:50.192 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7200 IOs, 3074966528 bytes 2026-02-19T20:42:50.238 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7300 IOs, 3091835904 bytes 2026-02-19T20:42:50.312 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7400 IOs, 3183774720 bytes 2026-02-19T20:42:50.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7500 IOs, 3208410112 bytes 2026-02-19T20:42:50.398 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2910832168556968) > weight (0.25) 2026-02-19T20:42:50.398 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7805397298968443) > weight (0.25) 2026-02-19T20:42:50.398 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4212399263112915) > weight (0.25) 2026-02-19T20:42:50.398 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8342893997895194) > weight (0.25) 2026-02-19T20:42:50.398 INFO:tasks.rbd_mirror_thrash:waiting for 96.77492365279659 secs before thrashing 2026-02-19T20:42:50.423 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7600 IOs, 3232675840 bytes 2026-02-19T20:42:50.496 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7700 IOs, 3309432832 bytes 2026-02-19T20:42:50.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7800 IOs, 3361551360 bytes 2026-02-19T20:42:50.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:7900 IOs, 3384551424 bytes 2026-02-19T20:42:50.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8000 IOs, 3423662080 bytes 2026-02-19T20:42:50.748 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8100 IOs, 3484034048 bytes 2026-02-19T20:42:50.835 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8200 IOs, 3585477632 bytes 2026-02-19T20:42:50.912 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8300 IOs, 3663325184 bytes 2026-02-19T20:42:50.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8400 IOs, 3711412224 bytes 2026-02-19T20:42:51.022 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8500 IOs, 3734344704 bytes 2026-02-19T20:42:51.079 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8600 IOs, 3775316992 bytes 2026-02-19T20:42:51.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8700 IOs, 3842595840 bytes 2026-02-19T20:42:51.233 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8800 IOs, 3910501376 bytes 2026-02-19T20:42:51.302 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:8900 IOs, 3969235968 bytes 2026-02-19T20:42:51.354 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9000 IOs, 3997887488 bytes 2026-02-19T20:42:51.398 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9100 IOs, 4043738112 bytes 2026-02-19T20:42:51.473 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9200 IOs, 4087608320 bytes 2026-02-19T20:42:51.527 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9300 IOs, 4114302976 bytes 2026-02-19T20:42:51.582 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9400 IOs, 4154421248 bytes 2026-02-19T20:42:51.639 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9500 IOs, 4183571456 bytes 2026-02-19T20:42:51.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9600 IOs, 4224618496 bytes 2026-02-19T20:42:51.729 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9700 IOs, 4225196032 bytes 2026-02-19T20:42:51.793 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9800 IOs, 4267016192 bytes 2026-02-19T20:42:51.858 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:9900 IOs, 4321224704 bytes 2026-02-19T20:42:51.907 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10000 IOs, 4333385728 bytes 2026-02-19T20:42:51.989 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10100 IOs, 4409165824 bytes 2026-02-19T20:42:52.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10200 IOs, 4430034944 bytes 2026-02-19T20:42:52.063 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10300 IOs, 4430657536 bytes 2026-02-19T20:42:52.115 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10400 IOs, 4462248960 bytes 2026-02-19T20:42:52.183 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10500 IOs, 4509523968 bytes 2026-02-19T20:42:52.220 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10600 IOs, 4525922304 bytes 2026-02-19T20:42:52.290 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10700 IOs, 4583193600 bytes 2026-02-19T20:42:52.348 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10800 IOs, 4623564800 bytes 2026-02-19T20:42:52.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:10900 IOs, 4670416896 bytes 2026-02-19T20:42:52.483 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11000 IOs, 4730774528 bytes 2026-02-19T20:42:52.556 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11100 IOs, 4794328064 bytes 2026-02-19T20:42:52.605 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11200 IOs, 4813798400 bytes 2026-02-19T20:42:52.635 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11300 IOs, 4814398464 bytes 2026-02-19T20:42:52.687 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11400 IOs, 4847030272 bytes 2026-02-19T20:42:52.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11500 IOs, 4897100800 bytes 2026-02-19T20:42:52.815 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11600 IOs, 4939424768 bytes 2026-02-19T20:42:52.891 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11700 IOs, 4997944320 bytes 2026-02-19T20:42:52.947 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11800 IOs, 5033193472 bytes 2026-02-19T20:42:53.005 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:11900 IOs, 5051802624 bytes 2026-02-19T20:42:53.093 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12000 IOs, 5128148992 bytes 2026-02-19T20:42:53.140 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12100 IOs, 5151388672 bytes 2026-02-19T20:42:53.182 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12200 IOs, 5187635200 bytes 2026-02-19T20:42:53.260 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12300 IOs, 5232949248 bytes 2026-02-19T20:42:53.340 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12400 IOs, 5319939072 bytes 2026-02-19T20:42:53.411 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12500 IOs, 5376567296 bytes 2026-02-19T20:42:53.491 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12600 IOs, 5437136896 bytes 2026-02-19T20:42:53.555 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12700 IOs, 5491296256 bytes 2026-02-19T20:42:53.611 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12800 IOs, 5526689792 bytes 2026-02-19T20:42:53.659 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:12900 IOs, 5553758208 bytes 2026-02-19T20:42:53.716 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13000 IOs, 5598932992 bytes 2026-02-19T20:42:53.767 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13100 IOs, 5639419904 bytes 2026-02-19T20:42:53.827 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13200 IOs, 5661287424 bytes 2026-02-19T20:42:53.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13300 IOs, 5690533888 bytes 2026-02-19T20:42:53.939 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13400 IOs, 5737480192 bytes 2026-02-19T20:42:53.996 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13500 IOs, 5777193984 bytes 2026-02-19T20:42:54.056 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13600 IOs, 5827235840 bytes 2026-02-19T20:42:54.117 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13700 IOs, 5853142016 bytes 2026-02-19T20:42:54.166 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13800 IOs, 5900623872 bytes 2026-02-19T20:42:54.256 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:13900 IOs, 5963261952 bytes 2026-02-19T20:42:54.309 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14000 IOs, 6001650688 bytes 2026-02-19T20:42:54.378 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14100 IOs, 6056629248 bytes 2026-02-19T20:42:54.440 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14200 IOs, 6112519168 bytes 2026-02-19T20:42:54.503 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14300 IOs, 6155074560 bytes 2026-02-19T20:42:54.562 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14400 IOs, 6204471296 bytes 2026-02-19T20:42:54.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14500 IOs, 6291142656 bytes 2026-02-19T20:42:54.728 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14600 IOs, 6357471232 bytes 2026-02-19T20:42:54.807 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14700 IOs, 6430262272 bytes 2026-02-19T20:42:54.859 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14800 IOs, 6468639744 bytes 2026-02-19T20:42:54.920 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:14900 IOs, 6503434240 bytes 2026-02-19T20:42:54.991 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15000 IOs, 6558725120 bytes 2026-02-19T20:42:55.058 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15100 IOs, 6614790144 bytes 2026-02-19T20:42:55.110 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15200 IOs, 6647316480 bytes 2026-02-19T20:42:55.160 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15300 IOs, 6676941824 bytes 2026-02-19T20:42:55.221 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15400 IOs, 6727419904 bytes 2026-02-19T20:42:55.270 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15500 IOs, 6746381312 bytes 2026-02-19T20:42:55.300 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15600 IOs, 6746987520 bytes 2026-02-19T20:42:55.346 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15700 IOs, 6769665024 bytes 2026-02-19T20:42:55.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15800 IOs, 6826540032 bytes 2026-02-19T20:42:55.449 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:15900 IOs, 6844979200 bytes 2026-02-19T20:42:55.564 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16000 IOs, 6954533888 bytes 2026-02-19T20:42:55.608 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16100 IOs, 6988310528 bytes 2026-02-19T20:42:55.700 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16200 IOs, 7074792448 bytes 2026-02-19T20:42:55.787 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16300 IOs, 7159508992 bytes 2026-02-19T20:42:55.850 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16400 IOs, 7206090752 bytes 2026-02-19T20:42:55.913 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16500 IOs, 7254274048 bytes 2026-02-19T20:42:55.964 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16600 IOs, 7281258496 bytes 2026-02-19T20:42:56.007 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16700 IOs, 7299884032 bytes 2026-02-19T20:42:56.040 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16800 IOs, 7303022592 bytes 2026-02-19T20:42:56.116 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:16900 IOs, 7378972672 bytes 2026-02-19T20:42:56.188 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17000 IOs, 7445294080 bytes 2026-02-19T20:42:56.244 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17100 IOs, 7471401984 bytes 2026-02-19T20:42:56.304 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17200 IOs, 7514312704 bytes 2026-02-19T20:42:56.339 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17300 IOs, 7519940608 bytes 2026-02-19T20:42:56.425 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17400 IOs, 7599690752 bytes 2026-02-19T20:42:56.469 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17500 IOs, 7628685312 bytes 2026-02-19T20:42:56.536 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17600 IOs, 7674771456 bytes 2026-02-19T20:42:56.606 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17700 IOs, 7727902720 bytes 2026-02-19T20:42:56.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17800 IOs, 7748578304 bytes 2026-02-19T20:42:56.697 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:17900 IOs, 7779540992 bytes 2026-02-19T20:42:56.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18000 IOs, 7813657600 bytes 2026-02-19T20:42:56.828 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18100 IOs, 7867758592 bytes 2026-02-19T20:42:56.890 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18200 IOs, 7898918912 bytes 2026-02-19T20:42:56.971 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18300 IOs, 7968205824 bytes 2026-02-19T20:42:57.037 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18400 IOs, 8026241024 bytes 2026-02-19T20:42:57.078 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18500 IOs, 8035709952 bytes 2026-02-19T20:42:57.121 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18600 IOs, 8053739520 bytes 2026-02-19T20:42:57.163 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18700 IOs, 8077004800 bytes 2026-02-19T20:42:57.248 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18800 IOs, 8151808000 bytes 2026-02-19T20:42:57.320 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:18900 IOs, 8218281984 bytes 2026-02-19T20:42:57.363 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19000 IOs, 8236003328 bytes 2026-02-19T20:42:57.419 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19100 IOs, 8274889728 bytes 2026-02-19T20:42:57.480 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19200 IOs, 8326486016 bytes 2026-02-19T20:42:57.544 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19300 IOs, 8380429312 bytes 2026-02-19T20:42:57.581 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19400 IOs, 8392449024 bytes 2026-02-19T20:42:57.616 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19500 IOs, 8412625920 bytes 2026-02-19T20:42:57.692 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:19600 IOs, 8476019712 bytes 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=124 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 124 -eq 124 ']' 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_10 present 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=present 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_10 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_10 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2026-02-19T20:42:57.713 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:42:57.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id=113a39d3a7a7 2026-02-19T20:42:57.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:42:57.740 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:42:57.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_10 present 113a39d3a7a7 2026-02-19T20:42:57.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:42:57.841 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=present 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id=113a39d3a7a7 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_10 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_10 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2026-02-19T20:42:57.842 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id=113a39d3a7a7 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 113a39d3a7a7 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -z 113a39d3a7a7 -o 113a39d3a7a7 = 113a39d3a7a7 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_state=present 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test present = present 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ disable_mirror cluster2 mirror image_10 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:42:57.868 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 mirror image disable mirror/image_10 2026-02-19T20:42:58.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:42:58.735+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5112800 scan_remote_mirror_snapshots: failed to locate remote start snapshot: snap_id=194 2026-02-19T20:42:58.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:42:58.735+0000 7f4388ef5640 -1 rbd::mirror::image_replayer::snapshot::Replayer: 0x5112800 scan_remote_mirror_snapshots: split-brain detected: failed to find matching non-primary snapshot in remote image: local_snap_id_start=194, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e7b2fa5d-8b41-4b57-afd6-1e94994e71e3, primary_snap_id=c2, last_copied_object_number=128, snap_seqs={194=18446744073709551614}] 2026-02-19T20:43:03.739 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:Mirroring disabled 2026-02-19T20:43:03.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_present cluster1 mirror image_10 deleted 2026-02-19T20:43:03.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:43:03.746 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local state=deleted 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local s 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_10 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_10 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2026-02-19T20:43:03.747 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:43:03.769 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_10: (2) No such file or directory 2026-02-19T20:43:03.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ image_id= 2026-02-19T20:43:03.770 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0.1 1 2 4 8 8 8 8 8 8 8 8 16 16 32 32 2026-02-19T20:43:03.771 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0.1 2026-02-19T20:43:03.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test_image_present cluster1 mirror image_10 deleted '' 2026-02-19T20:43:03.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:43:03.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:03.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:03.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local test_state=deleted 2026-02-19T20:43:03.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image_id= 2026-02-19T20:43:03.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_state=deleted 2026-02-19T20:43:03.871 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local current_image_id 2026-02-19T20:43:03.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ get_image_id cluster1 mirror image_10 2026-02-19T20:43:03.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:43:03.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:03.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local image=image_10 2026-02-19T20:43:03.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rbd --cluster=cluster1 info mirror/image_10 2026-02-19T20:43:03.872 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ sed -ne 's/^.*block_name_prefix: rbd_data\.//p' 2026-02-19T20:43:03.895 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error opening image image_10: (2) No such file or directory 2026-02-19T20:43:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ current_image_id= 2026-02-19T20:43:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n '' 2026-02-19T20:43:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test deleted = deleted 2026-02-19T20:43:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:43:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ purge_snapshots cluster2 mirror image_10 2026-02-19T20:43:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:03.897 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster cluster2 snap purge mirror/image_10 2026-02-19T20:43:03.923 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image_retry cluster2 mirror image_10 2026-02-19T20:43:03.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:03.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:03.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:03.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:43:03.924 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:43:03.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_10 2026-02-19T20:43:03.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:03.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:03.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:03.925 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_10 2026-02-19T20:43:03.950 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_10 2026-02-19T20:43:03.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:43:03.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:43:03.975 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:43:03.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:43:03.976 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 1 2026-02-19T20:43:04.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_10 2026-02-19T20:43:04.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:04.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:04.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:04.978 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_10 2026-02-19T20:43:05.006 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_10 2026-02-19T20:43:05.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:43:05.031 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:43:05.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:43:05.032 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:43:05.033 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 2 2026-02-19T20:43:07.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_10 2026-02-19T20:43:07.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:07.034 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:07.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:07.035 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_10 2026-02-19T20:43:07.062 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_10 2026-02-19T20:43:07.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:43:07.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:43:07.087 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:43:07.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:43:07.088 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 4 2026-02-19T20:43:11.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_10 2026-02-19T20:43:11.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:11.090 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:11.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:11.091 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_10 2026-02-19T20:43:11.323 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_10 2026-02-19T20:43:11.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:43:11.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:43:11.349 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:43:11.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:43:11.350 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 8 2026-02-19T20:43:19.351 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_10 2026-02-19T20:43:19.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:19.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:19.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:19.352 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_10 2026-02-19T20:43:19.380 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_10 2026-02-19T20:43:19.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 0% complete...failed. 2026-02-19T20:43:19.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:rbd: error: image still has watchers 2026-02-19T20:43:19.406 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:This means the image is still open or the client using it crashed. Try again after closing/unmapping it or waiting 30s for the crashed client to timeout. 2026-02-19T20:43:19.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 4 8 16 32 2026-02-19T20:43:19.407 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 16 2026-02-19T20:43:35.408 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ remove_image cluster2 mirror image_10 2026-02-19T20:43:35.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:35.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:35.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local image=image_10 2026-02-19T20:43:35.409 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 snap purge mirror/image_10 2026-02-19T20:43:35.643 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rbd --cluster=cluster2 rm mirror/image_10 2026-02-19T20:43:35.753 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2026-02-19T20:43:35.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:43:35.755 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ testlog 'TEST: check if removed images'\'' OMAP are removed' 2026-02-19T20:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ tee -a /tmp/tmp.9BL5XQhnTi/rbd-mirror.test.log 2026-02-19T20:43:35.756 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ date '+%F %T' 2026-02-19T20:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo 2026-02-19 20:43:35 TEST: check if removed 'images'\''' OMAP are removed 2026-02-19T20:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:2026-02-19 20:43:35 TEST: check if removed images' OMAP are removed 2026-02-19T20:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_in_omap cluster1 mirror 2026-02-19T20:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_omap_keys cluster1 mirror rbd_mirroring status_global 2026-02-19T20:43:35.757 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:43:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local obj_name=rbd_mirroring 2026-02-19T20:43:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local filter=status_global 2026-02-19T20:43:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2026-02-19T20:43:35.758 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:43:35.759 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ count_omap_keys_with_filter cluster1 mirror rbd_mirroring status_global 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirroring 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local filter=status_global 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ list_omap_keys cluster1 mirror rbd_mirroring 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirroring 2026-02-19T20:43:35.760 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rados --cluster cluster1 -p mirror listomapkeys rbd_mirroring 2026-02-19T20:43:35.761 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -c status_global 2026-02-19T20:43:35.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 = 0 2026-02-19T20:43:35.780 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 0 ']' 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_omap_keys cluster1 mirror rbd_mirroring image_ 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local obj_name=rbd_mirroring 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local filter=image_ 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2026-02-19T20:43:35.781 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:43:35.782 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:43:35.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ count_omap_keys_with_filter cluster1 mirror rbd_mirroring image_ 2026-02-19T20:43:35.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:43:35.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirroring 2026-02-19T20:43:35.783 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local filter=image_ 2026-02-19T20:43:35.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ list_omap_keys cluster1 mirror rbd_mirroring 2026-02-19T20:43:35.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:43:35.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirroring 2026-02-19T20:43:35.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rados --cluster cluster1 -p mirror listomapkeys rbd_mirroring 2026-02-19T20:43:35.784 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -c image_ 2026-02-19T20:43:35.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 = 0 2026-02-19T20:43:35.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 0 ']' 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_omap_keys cluster1 mirror rbd_mirror_leader image_map 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local obj_name=rbd_mirror_leader 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local filter=image_map 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:43:35.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ count_omap_keys_with_filter cluster1 mirror rbd_mirror_leader image_map 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirror_leader 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local filter=image_map 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ list_omap_keys cluster1 mirror rbd_mirror_leader 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster1 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirror_leader 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rados --cluster cluster1 -p mirror listomapkeys rbd_mirror_leader 2026-02-19T20:43:35.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -c image_map 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 = 0 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 0 ']' 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_image_in_omap cluster2 mirror 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_omap_keys cluster2 mirror rbd_mirroring status_global 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local obj_name=rbd_mirroring 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local filter=status_global 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:43:35.822 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ count_omap_keys_with_filter cluster2 mirror rbd_mirroring status_global 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster2 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirroring 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local filter=status_global 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ list_omap_keys cluster2 mirror rbd_mirroring 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster2 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirroring 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rados --cluster cluster2 -p mirror listomapkeys rbd_mirroring 2026-02-19T20:43:35.823 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -c status_global 2026-02-19T20:43:35.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 = 0 2026-02-19T20:43:35.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:43:35.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:43:35.843 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 0 ']' 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_omap_keys cluster2 mirror rbd_mirroring image_ 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local obj_name=rbd_mirroring 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local filter=image_ 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ count_omap_keys_with_filter cluster2 mirror rbd_mirroring image_ 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster2 2026-02-19T20:43:35.844 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirroring 2026-02-19T20:43:35.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local filter=image_ 2026-02-19T20:43:35.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ list_omap_keys cluster2 mirror rbd_mirroring 2026-02-19T20:43:35.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster2 2026-02-19T20:43:35.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirroring 2026-02-19T20:43:35.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rados --cluster cluster2 -p mirror listomapkeys rbd_mirroring 2026-02-19T20:43:35.845 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -c image_ 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 = 0 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 0 ']' 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ wait_for_omap_keys cluster2 mirror rbd_mirror_leader image_map 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local pool=mirror 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local obj_name=rbd_mirror_leader 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local filter=image_map 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for s in 0 1 2 2 4 4 8 8 8 16 16 32 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ sleep 0 2026-02-19T20:43:35.861 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ count_omap_keys_with_filter cluster2 mirror rbd_mirror_leader image_map 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster2 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirror_leader 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local filter=image_map 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ list_omap_keys cluster2 mirror rbd_mirror_leader 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local cluster=cluster2 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ grep -c image_map 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local pool=mirror 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ local obj_name=rbd_mirror_leader 2026-02-19T20:43:35.862 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ rados --cluster cluster2 -p mirror listomapkeys rbd_mirror_leader 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test 0 = 0 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ error_code=0 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set -e 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 0 ']' 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 0 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ cleanup 0 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local error_code=0 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ set +e 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -ne 0 ']' 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -z '' ']' 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster instance 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:43:35.880 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ ceph --cluster cluster1 osd pool rm mirror mirror --yes-i-really-really-mean-it 2026-02-19T20:43:36.646 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:pool 'mirror' does not exist 2026-02-19T20:43:36.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:43:36.654 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ ceph --cluster cluster2 osd pool rm mirror mirror --yes-i-really-really-mean-it 2026-02-19T20:43:37.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.535+0000 7f563a2f0640 -1 rbd::mirror::InstanceWatcher: 0x40d1880 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5897 object: (107) Transport endpoint is not connected 2026-02-19T20:43:37.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.535+0000 7f563a2f0640 -1 rbd::mirror::InstanceWatcher: 0x40d0380 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5897 object: (107) Transport endpoint is not connected 2026-02-19T20:43:37.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.535+0000 7f563a2f0640 -1 rbd::mirror::InstanceWatcher: 0x40d0a80 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5897 object: (107) Transport endpoint is not connected 2026-02-19T20:43:37.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.535+0000 7f5633ae3640 -1 rbd::mirror::InstanceWatcher: 0x40d1880 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:37.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.535+0000 7f5633ae3640 -1 rbd::mirror::InstanceWatcher: 0x40d0a80 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:37.538 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.535+0000 7f56342e4640 -1 rbd::mirror::InstanceWatcher: 0x40d0380 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:37.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.536+0000 7f5633ae3640 -1 rbd::mirror::MirrorStatusUpdater 0x414a000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:37.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.536+0000 7f563aaf1640 -1 rbd::mirror::NamespaceReplayer: 0x30596c0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:37.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.537+0000 7f56342e4640 -1 rbd::mirror::MirrorStatusUpdater 0x414a240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:37.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.537+0000 7f563aaf1640 -1 rbd::mirror::NamespaceReplayer: 0x3059340 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:37.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.537+0000 7f5633ae3640 -1 rbd::mirror::MirrorStatusUpdater 0x414a120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:37.540 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.537+0000 7f563aaf1640 -1 rbd::mirror::NamespaceReplayer: 0x3059500 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:37.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:37.755+0000 7f563aaf1640 -1 rbd::mirror::LeaderWatcher: 0x4d21200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:37.824 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:pool 'mirror' does not exist 2026-02-19T20:43:37.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:43:37.838 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ ceph --cluster cluster1 osd pool rm mirror_parent mirror_parent --yes-i-really-really-mean-it 2026-02-19T20:43:38.650 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:pool 'mirror_parent' does not exist 2026-02-19T20:43:38.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ CEPH_ARGS= 2026-02-19T20:43:38.658 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ ceph --cluster cluster2 osd pool rm mirror_parent mirror_parent --yes-i-really-really-mean-it 2026-02-19T20:43:39.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.218+0000 7f34b5e73640 -1 rbd::mirror::RemotePoolPoller: 0x3083ee0 handle_mirror_peer_ping: failed to ping remote mirror peer: (2) No such file or directory 2026-02-19T20:43:39.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.218+0000 7f34b5672640 -1 rbd::mirror::RemotePoolPoller: 0x3083ee0 handle_mirror_peer_list: failed to retrieve mirror peers: (2) No such file or directory 2026-02-19T20:43:39.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.218+0000 7f34b5672640 -1 rbd::mirror::PoolReplayer: 0x3093500 handle_remote_pool_meta_updated: remote pool metadata updated unexpectedly 2026-02-19T20:43:39.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34c268c640 -1 rbd::mirror::InstanceWatcher: 0x42c4000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5859 object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34c2e8d640 -1 rbd::mirror::InstanceWatcher: 0x40c3180 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5859 object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34c2e8d640 -1 rbd::mirror::InstanceWatcher: 0x40c3880 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5859 object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34bc680640 -1 rbd::mirror::InstanceWatcher: 0x42c4000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34b5e73640 -1 rbd::mirror::MirrorStatusUpdater 0x41865a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34c2e8d640 -1 rbd::mirror::NamespaceReplayer: 0x309b6c0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34bbe7f640 -1 rbd::mirror::MirrorStatusUpdater 0x4186000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34c268c640 -1 rbd::mirror::NamespaceReplayer: 0x309b6c0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34bc680640 -1 rbd::mirror::InstanceWatcher: 0x40c3180 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34bbe7f640 -1 rbd::mirror::InstanceWatcher: 0x40c3880 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34b5672640 -1 rbd::mirror::MirrorStatusUpdater 0x4186360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34c268c640 -1 rbd::mirror::NamespaceReplayer: 0x309b500 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34bc680640 -1 rbd::mirror::MirrorStatusUpdater 0x4186120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34c268c640 -1 rbd::mirror::NamespaceReplayer: 0x309b500 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34b5e73640 -1 rbd::mirror::MirrorStatusUpdater 0x4186480 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34c2e8d640 -1 rbd::mirror::NamespaceReplayer: 0x309b340 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34bbe7f640 -1 rbd::mirror::MirrorStatusUpdater 0x4186240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:39.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.219+0000 7f34c268c640 -1 rbd::mirror::NamespaceReplayer: 0x309b340 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:39.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.271+0000 7f34c268c640 -1 rbd::mirror::InstanceWatcher: 0x478c000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5861 object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.271+0000 7f34c268c640 -1 rbd::mirror::InstanceWatcher: 0x3e99c00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5861 object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.271+0000 7f34aee65640 -1 rbd::mirror::InstanceWatcher: 0x478c000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:39.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.271+0000 7f34ae664640 -1 rbd::mirror::InstanceWatcher: 0x3e99c00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:39.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.272+0000 7f34aee65640 -1 rbd::mirror::MirrorStatusUpdater 0x43d30e0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:39.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.272+0000 7f34c268c640 -1 rbd::mirror::NamespaceReplayer: 0x309ae00 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:39.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.273+0000 7f34ae664640 -1 rbd::mirror::MirrorStatusUpdater 0x43d3200 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:39.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.273+0000 7f34c268c640 -1 rbd::mirror::NamespaceReplayer: 0x309b180 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:39.279 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.277+0000 7f34c2e8d640 -1 rbd::mirror::LeaderWatcher: 0x4d73200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:39.278+0000 7f34c2e8d640 -1 rbd::mirror::LeaderWatcher: 0x445f200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.619+0000 7f439570e640 -1 rbd::mirror::LeaderWatcher: 0x4a51200 handle_notify_heartbeat: error notifying heartbeat: (2) No such file or directory, releasing leader 2026-02-19T20:43:39.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.619+0000 7f438ef01640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x7b7f200 handle_unregister_watcher: error unregistering watcher for trash directory: (107) Transport endpoint is not connected 2026-02-19T20:43:39.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.619+0000 7f438ef01640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x7b7e480 handle_unregister_watcher: error unregistering watcher for trash directory: (107) Transport endpoint is not connected 2026-02-19T20:43:39.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.619+0000 7f438ef01640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x7b7e000 handle_unregister_watcher: error unregistering watcher for trash directory: (107) Transport endpoint is not connected 2026-02-19T20:43:39.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.619+0000 7f438ef01640 -1 rbd::mirror::PoolWatcher: 0x2d61680 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.619+0000 7f43886f4640 -1 rbd::mirror::PoolWatcher: 0x512c3c0 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.619+0000 7f43886f4640 -1 rbd::mirror::PoolWatcher: 0x512c5a0 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.619+0000 7f438f702640 -1 rbd::mirror::PoolWatcher: 0x2d605a0 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.620+0000 7f438f702640 -1 rbd::mirror::PoolWatcher: 0x512c960 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.620+0000 7f4388ef5640 -1 rbd::mirror::PoolWatcher: 0x512c780 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:39.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.620+0000 7f4395f0f640 -1 rbd::mirror::LeaderWatcher: 0x4a51200 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2026-02-19T20:43:39.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:39.620+0000 7f4395f0f640 -1 rbd::mirror::LeaderWatcher: 0x4a51200 handle_release_leader_lock: error releasing lock: (2) No such file or directory 2026-02-19T20:43:39.792 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:pool 'mirror_parent' does not exist 2026-02-19T20:43:39.800 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for cluster in "${CLUSTER1}" "${CLUSTER2}" 2026-02-19T20:43:39.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirrors cluster1 2026-02-19T20:43:39.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1 2026-02-19T20:43:39.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.801 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 0 3 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirror cluster1:0 '' 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1:0 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirror cluster1:1 '' 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1:1 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.802 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirror cluster1:2 '' 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1:2 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirror cluster1:3 '' 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster1:3 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for cluster in "${CLUSTER1}" "${CLUSTER2}" 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirrors cluster2 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.803 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:++ seq 0 3 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirror cluster2:0 '' 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2:0 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirror cluster2:1 '' 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2:1 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirror cluster2:2 '' 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2:2 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ for instance in `seq 0 ${LAST_MIRROR_INSTANCE}` 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ stop_mirror cluster2:3 '' 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local cluster=cluster2:3 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ local sig= 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test -n 1 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ return 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' -z 1 ']' 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ test '' = /tmp/tmp.9BL5XQhnTi 2026-02-19T20:43:39.804 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ rm -Rf /tmp/tmp.9BL5XQhnTi 2026-02-19T20:43:39.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ '[' 0 -eq 0 ']' 2026-02-19T20:43:39.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ echo OK 2026-02-19T20:43:39.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stdout:OK 2026-02-19T20:43:39.806 INFO:tasks.workunit.cluster1.client.mirror.trial080.stderr:+ exit 0 2026-02-19T20:43:39.806 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T20:43:39.806 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp 2026-02-19T20:43:39.874 INFO:tasks.workunit:Stopping ['rbd/rbd_mirror_stress.sh'] on cluster1.client.mirror... 2026-02-19T20:43:39.874 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.cluster1.client.mirror /home/ubuntu/cephtest/clone.cluster1.client.mirror 2026-02-19T20:43:40.230 DEBUG:teuthology.parallel:result is None 2026-02-19T20:43:40.230 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2026-02-19T20:43:40.257 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2026-02-19T20:43:40.257 DEBUG:teuthology.orchestra.run.trial080:> rmdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2026-02-19T20:43:40.314 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror 2026-02-19T20:43:40.315 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror-thrash 2026-02-19T20:43:40.331 INFO:tasks.rbd_mirror_thrash:joining rbd_mirror_thrash 2026-02-19T20:43:40.332 INFO:tasks.rbd_mirror_thrash:stat['kill'] = 11 2026-02-19T20:43:40.332 INFO:tasks.rbd_mirror_thrash:done joining 2026-02-19T20:43:40.332 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror-thrash 2026-02-19T20:43:40.348 INFO:tasks.rbd_mirror_thrash:joining rbd_mirror_thrash 2026-02-19T20:43:40.348 INFO:tasks.rbd_mirror_thrash:stat['kill'] = 10 2026-02-19T20:43:40.349 INFO:tasks.rbd_mirror_thrash:done joining 2026-02-19T20:43:40.349 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2026-02-19T20:43:40.366 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.3:waiting for process to exit 2026-02-19T20:43:40.366 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:43:40.370 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd7af9640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 48090) UID: 1000 2026-02-19T20:43:40.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x4ddb200 handle_unregister_watcher: error unregistering watcher for trash directory: (107) Transport endpoint is not connected 2026-02-19T20:43:40.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x4ddb440 handle_unregister_watcher: error unregistering watcher for trash directory: (107) Transport endpoint is not connected 2026-02-19T20:43:40.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x4ddbb00 handle_unregister_watcher: error unregistering watcher for trash directory: (107) Transport endpoint is not connected 2026-02-19T20:43:40.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd02ea640 -1 rbd::mirror::PoolWatcher: 0x4b792c0 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fca2de640 -1 rbd::mirror::PoolWatcher: 0x4b794a0 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::PoolWatcher: 0x4b790e0 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd02ea640 -1 rbd::mirror::PoolWatcher: 0x4b78d20 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc9add640 -1 rbd::mirror::PoolWatcher: 0x4b79e00 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc9add640 -1 rbd::mirror::PoolWatcher: 0x4b79a40 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc32d0640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x4a6bd40 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc32d0640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x4d7fd40 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc2acf640 -1 rbd::mirror::PoolWatcher: 0x4b783c0 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fbc2c2640 -1 rbd::mirror::PoolWatcher: 0x4b78960 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc2acf640 -1 rbd::mirror::PoolWatcher: 0x4b785a0 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fbcac3640 -1 rbd::mirror::PoolWatcher: 0x4b78780 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::InstanceWatcher: 0x429b500 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4305 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::InstanceWatcher: 0x420e000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4305 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::InstanceWatcher: 0x429aa80 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4305 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::InstanceWatcher: 0x429bc00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4325 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::InstanceWatcher: 0x429b180 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4325 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::InstanceWatcher: 0x429b500 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::InstanceWatcher: 0x420e000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::InstanceWatcher: 0x429aa80 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fca2de640 -1 rbd::mirror::MirrorStatusUpdater 0x3edd440 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fca2de640 -1 rbd::mirror::MirrorStatusUpdater 0x3edd560 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fca2de640 -1 rbd::mirror::MirrorStatusUpdater 0x3edd680 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::NamespaceReplayer: 0x2acc380 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::NamespaceReplayer: 0x2acce00 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::NamespaceReplayer: 0x2acc000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::MirrorStatusUpdater 0x3edcd80 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::MirrorStatusUpdater 0x3edcfc0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd0aeb640 -1 rbd::mirror::MirrorStatusUpdater 0x3edcea0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.373 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd6af7640 -1 rbd::mirror::NamespaceReplayer: 0x2acc380 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd6af7640 -1 rbd::mirror::NamespaceReplayer: 0x2acce00 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd6af7640 -1 rbd::mirror::NamespaceReplayer: 0x2acc000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc32d0640 -1 rbd::mirror::InstanceWatcher: 0x429bc00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc32d0640 -1 rbd::mirror::InstanceWatcher: 0x429b180 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fbc2c2640 -1 rbd::mirror::MirrorStatusUpdater 0x3edd7a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fbc2c2640 -1 rbd::mirror::MirrorStatusUpdater 0x3edd8c0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::NamespaceReplayer: 0x424a000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::NamespaceReplayer: 0x2acd180 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc32d0640 -1 rbd::mirror::MirrorStatusUpdater 0x3edd200 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fc32d0640 -1 rbd::mirror::MirrorStatusUpdater 0x3edd0e0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::NamespaceReplayer: 0x424a000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.364+0000 7f5fd72f8640 -1 rbd::mirror::NamespaceReplayer: 0x2acd180 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.365+0000 7f5fd72f8640 -1 rbd::mirror::LeaderWatcher: 0x3bea300 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2026-02-19T20:43:40.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.365+0000 7f5fd72f8640 -1 rbd::mirror::LeaderWatcher: 0x3bea300 handle_shut_down_leader_lock: error shutting down leader lock: (2) No such file or directory 2026-02-19T20:43:40.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.365+0000 7f5fd6af7640 -1 rbd::mirror::LeaderWatcher: 0x3bea300 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.367+0000 7f5fd72f8640 -1 rbd::mirror::LeaderWatcher: 0x3a31b00 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2026-02-19T20:43:40.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.367+0000 7f5fd72f8640 -1 rbd::mirror::LeaderWatcher: 0x3a31b00 handle_shut_down_leader_lock: error shutting down leader lock: (2) No such file or directory 2026-02-19T20:43:40.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.367+0000 7f5fd6af7640 -1 rbd::mirror::LeaderWatcher: 0x3a31b00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.380 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Stopped 2026-02-19T20:43:40.381 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2026-02-19T20:43:40.398 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.2:waiting for process to exit 2026-02-19T20:43:40.398 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:43:40.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2639dd4640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 84195) UID: 1000 2026-02-19T20:43:40.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2638dd2640 -1 rbd::mirror::InstanceWatcher: 0x39bce00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6934 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2638dd2640 -1 rbd::mirror::InstanceWatcher: 0x39bd500 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6934 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2638dd2640 -1 rbd::mirror::InstanceWatcher: 0x39bca80 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6934 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f26255ab640 -1 rbd::mirror::InstanceWatcher: 0x39bd880 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2624daa640 -1 rbd::mirror::InstanceWatcher: 0x39cdc00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2632dc6640 -1 rbd::mirror::InstanceWatcher: 0x39bce00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2632dc6640 -1 rbd::mirror::InstanceWatcher: 0x39bca80 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2632dc6640 -1 rbd::mirror::InstanceWatcher: 0x39bd500 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f262c5b9640 -1 rbd::mirror::MirrorStatusUpdater 0x388c5a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f262c5b9640 -1 rbd::mirror::MirrorStatusUpdater 0x388c360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f262c5b9640 -1 rbd::mirror::MirrorStatusUpdater 0x388c480 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2638dd2640 -1 rbd::mirror::NamespaceReplayer: 0x27516c0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2638dd2640 -1 rbd::mirror::NamespaceReplayer: 0x2751340 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2638dd2640 -1 rbd::mirror::NamespaceReplayer: 0x2751500 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f26325c5640 -1 rbd::mirror::MirrorStatusUpdater 0x388c000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f26325c5640 -1 rbd::mirror::MirrorStatusUpdater 0x388c240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f2632dc6640 -1 rbd::mirror::MirrorStatusUpdater 0x388c120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.426 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f26395d3640 -1 rbd::mirror::NamespaceReplayer: 0x27516c0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.426 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f26395d3640 -1 rbd::mirror::NamespaceReplayer: 0x2751340 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.426 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.419+0000 7f26395d3640 -1 rbd::mirror::NamespaceReplayer: 0x2751500 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.426 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.421+0000 7f2638dd2640 -1 rbd::mirror::LeaderWatcher: 0x442f200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.433 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Stopped 2026-02-19T20:43:40.433 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2026-02-19T20:43:40.454 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.1:waiting for process to exit 2026-02-19T20:43:40.454 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:43:40.474 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6beefb5640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 81660) UID: 1000 2026-02-19T20:43:40.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bee7b4640 -1 rbd::mirror::InstanceWatcher: 0x2d84e00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6669 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bee7b4640 -1 rbd::mirror::InstanceWatcher: 0x1b33c00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6669 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bd9f8b640 -1 rbd::mirror::InstanceWatcher: 0x2d85500 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bee7b4640 -1 rbd::mirror::InstanceWatcher: 0x2d84700 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6669 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bda78c640 -1 rbd::mirror::InstanceWatcher: 0x2bf0a80 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6be77a6640 -1 rbd::mirror::InstanceWatcher: 0x2d84e00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6be7fa7640 -1 rbd::mirror::InstanceWatcher: 0x1b33c00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6be7fa7640 -1 rbd::mirror::InstanceWatcher: 0x2d84700 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6be0f99640 -1 rbd::mirror::MirrorStatusUpdater 0x2c4a360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bedfb3640 -1 rbd::mirror::NamespaceReplayer: 0x1b2d6c0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6be0f99640 -1 rbd::mirror::MirrorStatusUpdater 0x3792000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6be0f99640 -1 rbd::mirror::MirrorStatusUpdater 0x2c4a480 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6be77a6640 -1 rbd::mirror::MirrorStatusUpdater 0x2c4a000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bee7b4640 -1 rbd::mirror::NamespaceReplayer: 0x1b2d340 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bee7b4640 -1 rbd::mirror::NamespaceReplayer: 0x1b2d500 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6be7fa7640 -1 rbd::mirror::MirrorStatusUpdater 0x2c4a240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bee7b4640 -1 rbd::mirror::NamespaceReplayer: 0x1b2d6c0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6be7fa7640 -1 rbd::mirror::MirrorStatusUpdater 0x2c4a120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bedfb3640 -1 rbd::mirror::NamespaceReplayer: 0x1b2d340 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bedfb3640 -1 rbd::mirror::NamespaceReplayer: 0x1b2d500 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bd377e640 -1 rbd::mirror::MirrorStatusUpdater 0x2c4a900 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2026-02-19T20:43:40.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bd377e640 -1 rbd::mirror::MirrorStatusUpdater 0x2eb5320 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2026-02-19T20:43:40.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bedfb3640 -1 rbd::mirror::NamespaceReplayer: 0x1b2c000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2026-02-19T20:43:40.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.471+0000 7f6bedfb3640 -1 rbd::mirror::NamespaceReplayer: 0x1b2ce00 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2026-02-19T20:43:40.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.473+0000 7f6bee7b4640 -1 rbd::mirror::LeaderWatcher: 0x37fd500 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Stopped 2026-02-19T20:43:40.486 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2026-02-19T20:43:40.504 DEBUG:tasks.rbd_mirror.cluster2.client.mirror.0:waiting for process to exit 2026-02-19T20:43:40.504 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:43:40.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a34a8640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 83096) UID: 1000 2026-02-19T20:43:40.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a24a6640 -1 rbd::mirror::InstanceWatcher: 0x36a5500 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6824 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a24a6640 -1 rbd::mirror::InstanceWatcher: 0x36a4000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6824 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a24a6640 -1 rbd::mirror::InstanceWatcher: 0x322e000 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6822 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a24a6640 -1 rbd::mirror::InstanceWatcher: 0x3209880 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6822 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a24a6640 -1 rbd::mirror::InstanceWatcher: 0x322e380 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.6822 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc78e47e640 -1 rbd::mirror::InstanceWatcher: 0x36a5500 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc79bc99640 -1 rbd::mirror::InstanceWatcher: 0x322e000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc78ec7f640 -1 rbd::mirror::InstanceWatcher: 0x36a4000 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc79c49a640 -1 rbd::mirror::InstanceWatcher: 0x3209880 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc79c49a640 -1 rbd::mirror::InstanceWatcher: 0x322e380 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc79548c640 -1 rbd::mirror::MirrorStatusUpdater 0x30de480 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a2ca7640 -1 rbd::mirror::NamespaceReplayer: 0x1fa76c0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc795c8d640 -1 rbd::mirror::MirrorStatusUpdater 0x30de360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc795c8d640 -1 rbd::mirror::MirrorStatusUpdater 0x3c12000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a2ca7640 -1 rbd::mirror::NamespaceReplayer: 0x1fa7500 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a2ca7640 -1 rbd::mirror::NamespaceReplayer: 0x1fa7340 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc79bc99640 -1 rbd::mirror::MirrorStatusUpdater 0x30de000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc79bc99640 -1 rbd::mirror::MirrorStatusUpdater 0x30de120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc79bc99640 -1 rbd::mirror::MirrorStatusUpdater 0x30de240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a24a6640 -1 rbd::mirror::NamespaceReplayer: 0x1fa76c0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a24a6640 -1 rbd::mirror::NamespaceReplayer: 0x1fa7500 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.524+0000 7fc7a24a6640 -1 rbd::mirror::NamespaceReplayer: 0x1fa7340 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.525+0000 7fc787c71640 -1 rbd::mirror::MirrorStatusUpdater 0x30de7e0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.525+0000 7fc787c71640 -1 rbd::mirror::MirrorStatusUpdater 0x33245a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.525+0000 7fc7a2ca7640 -1 rbd::mirror::NamespaceReplayer: 0x1fa6000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.525+0000 7fc7a2ca7640 -1 rbd::mirror::NamespaceReplayer: 0x1fa6e00 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.525+0000 7fc78e47e640 -1 rbd::mirror::MirrorStatusUpdater 0x3325320 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.525+0000 7fc78e47e640 -1 rbd::mirror::MirrorStatusUpdater 0x3325440 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.525+0000 7fc7a24a6640 -1 rbd::mirror::NamespaceReplayer: 0x1fa6000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.525+0000 7fc7a24a6640 -1 rbd::mirror::NamespaceReplayer: 0x1fa6e00 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.525+0000 7fc7a2ca7640 -1 rbd::mirror::LeaderWatcher: 0x33ab200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.533 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.526+0000 7fc7a2ca7640 -1 rbd::mirror::LeaderWatcher: 0x3c79200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Stopped 2026-02-19T20:43:40.538 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2026-02-19T20:43:40.559 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.3:waiting for process to exit 2026-02-19T20:43:40.559 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:43:40.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f4396710640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 51826) UID: 1000 2026-02-19T20:43:40.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f43816e6640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x561a240 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2026-02-19T20:43:40.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f43816e6640 -1 rbd::mirror::image_deleter::TrashWatcher: 0x5439b00 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory 2026-02-19T20:43:40.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f4381ee7640 -1 rbd::mirror::PoolWatcher: 0x512cf00 operator(): error unregistering watcher for rbd_mirroring object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f439570e640 -1 rbd::mirror::InstanceWatcher: 0x3eb4380 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4410 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f439570e640 -1 rbd::mirror::InstanceWatcher: 0x3fd9180 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4410 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f439570e640 -1 rbd::mirror::InstanceWatcher: 0x3fd8a80 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4410 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f437b6da640 -1 rbd::mirror::PoolWatcher: 0x512d2c0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f437b6da640 -1 rbd::mirror::PoolWatcher: 0x512dc20 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f4395f0f640 -1 rbd::mirror::LeaderWatcher: 0x415d200 handle_notify_heartbeat: error notifying heartbeat: (2) No such file or directory, releasing leader 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.577+0000 7f4381ee7640 -1 rbd::mirror::PoolWatcher: 0x512d0e0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f438ef01640 -1 rbd::mirror::InstanceWatcher: 0x3eb4380 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f438ef01640 -1 rbd::mirror::InstanceWatcher: 0x3fd8a80 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f438ef01640 -1 rbd::mirror::InstanceWatcher: 0x3fd9180 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f4395f0f640 -1 rbd::mirror::InstanceWatcher: 0x4428a80 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4412 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.584 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f43886f4640 -1 rbd::mirror::MirrorStatusUpdater 0x3e8a5a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f43886f4640 -1 rbd::mirror::MirrorStatusUpdater 0x3e8a480 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f43886f4640 -1 rbd::mirror::MirrorStatusUpdater 0x3e8a360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f439570e640 -1 rbd::mirror::NamespaceReplayer: 0x2d936c0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f439570e640 -1 rbd::mirror::NamespaceReplayer: 0x2d93340 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f438f702640 -1 rbd::mirror::MirrorStatusUpdater 0x3e8a000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f439570e640 -1 rbd::mirror::NamespaceReplayer: 0x2d93500 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f438f702640 -1 rbd::mirror::MirrorStatusUpdater 0x3e8a240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f439570e640 -1 rbd::mirror::NamespaceReplayer: 0x2d936c0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f438f702640 -1 rbd::mirror::MirrorStatusUpdater 0x3e8a120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f439570e640 -1 rbd::mirror::NamespaceReplayer: 0x2d93340 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f439570e640 -1 rbd::mirror::NamespaceReplayer: 0x2d93500 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f439570e640 -1 rbd::mirror::InstanceWatcher: 0x4456700 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4412 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f43816e6640 -1 rbd::mirror::InstanceWatcher: 0x4428a80 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f437aed9640 -1 rbd::mirror::MirrorStatusUpdater 0x40d3440 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f439570e640 -1 rbd::mirror::NamespaceReplayer: 0x2d92000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f4381ee7640 -1 rbd::mirror::MirrorStatusUpdater 0x40d30e0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f4395f0f640 -1 rbd::mirror::NamespaceReplayer: 0x2d92000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f43816e6640 -1 rbd::mirror::InstanceWatcher: 0x4456700 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f437b6da640 -1 rbd::mirror::MirrorStatusUpdater 0x40d37a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f439570e640 -1 rbd::mirror::NamespaceReplayer: 0x2d92e00 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f4381ee7640 -1 rbd::mirror::MirrorStatusUpdater 0x40d3200 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f4395f0f640 -1 rbd::mirror::NamespaceReplayer: 0x2d92e00 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f4395f0f640 -1 rbd::mirror::LeaderWatcher: 0x415d200 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory 2026-02-19T20:43:40.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f4395f0f640 -1 rbd::mirror::LeaderWatcher: 0x415d200 handle_release_leader_lock: error releasing lock: (2) No such file or directory 2026-02-19T20:43:40.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.578+0000 7f4395f0f640 -1 rbd::mirror::LeaderWatcher: 0x415d200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.trial080.stderr:2026-02-19T20:43:40.580+0000 7f4395f0f640 -1 rbd::mirror::LeaderWatcher: 0x4a51200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Stopped 2026-02-19T20:43:40.595 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2026-02-19T20:43:40.613 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.2:waiting for process to exit 2026-02-19T20:43:40.613 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:43:40.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.633+0000 7f563b2f2640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 83370) UID: 1000 2026-02-19T20:43:40.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.633+0000 7f563a2f0640 -1 rbd::mirror::InstanceWatcher: 0x46c2700 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5900 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.633+0000 7f563a2f0640 -1 rbd::mirror::InstanceWatcher: 0x471c700 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5900 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f56262c8640 -1 rbd::mirror::InstanceWatcher: 0x46c2700 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f56262c8640 -1 rbd::mirror::InstanceWatcher: 0x471c700 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f561fabb640 -1 rbd::mirror::MirrorStatusUpdater 0x439b440 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f561fabb640 -1 rbd::mirror::MirrorStatusUpdater 0x439b560 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f563a2f0640 -1 rbd::mirror::NamespaceReplayer: 0x3058000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f563a2f0640 -1 rbd::mirror::NamespaceReplayer: 0x3058e00 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f5626ac9640 -1 rbd::mirror::MirrorStatusUpdater 0x439b0e0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f5626ac9640 -1 rbd::mirror::MirrorStatusUpdater 0x439b200 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f563aaf1640 -1 rbd::mirror::NamespaceReplayer: 0x3058000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f563a2f0640 -1 rbd::mirror::NamespaceReplayer: 0x3058e00 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.638 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.trial080.stderr:2026-02-19T20:43:40.634+0000 7f563a2f0640 -1 rbd::mirror::LeaderWatcher: 0x4421200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Stopped 2026-02-19T20:43:40.645 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2026-02-19T20:43:40.656 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.1:waiting for process to exit 2026-02-19T20:43:40.656 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:43:40.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.684+0000 7f707cd7d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 80127) UID: 1000 2026-02-19T20:43:40.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.684+0000 7f707c57c640 -1 rbd::mirror::InstanceWatcher: 0x3f7ee00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5805 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.684+0000 7f707c57c640 -1 rbd::mirror::InstanceWatcher: 0x3f7fc00 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5805 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.684+0000 7f707c57c640 -1 rbd::mirror::InstanceWatcher: 0x3a57180 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5803 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.684+0000 7f707c57c640 -1 rbd::mirror::InstanceWatcher: 0x36ab880 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5803 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.684+0000 7f707c57c640 -1 rbd::mirror::InstanceWatcher: 0x3a57880 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5803 object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f7068554640 -1 rbd::mirror::InstanceWatcher: 0x3f7ee00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f7067d53640 -1 rbd::mirror::InstanceWatcher: 0x3f7fc00 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f7061d47640 -1 rbd::mirror::MirrorStatusUpdater 0x3c197a0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2026-02-19T20:43:40.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707556e640 -1 rbd::mirror::InstanceWatcher: 0x3a57180 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.691 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f7061d47640 -1 rbd::mirror::MirrorStatusUpdater 0x3c19440 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (2) No such file or directory 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707556e640 -1 rbd::mirror::InstanceWatcher: 0x36ab880 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707556e640 -1 rbd::mirror::InstanceWatcher: 0x3a57880 handle_unregister_instance: error unregistering instance: (2) No such file or directory 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707bd7b640 -1 rbd::mirror::NamespaceReplayer: 0x28b4000 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707bd7b640 -1 rbd::mirror::NamespaceReplayer: 0x28b4e00 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (2) No such file or directory 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f7068554640 -1 rbd::mirror::MirrorStatusUpdater 0x3c190e0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f7067d53640 -1 rbd::mirror::MirrorStatusUpdater 0x3c19200 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707bd7b640 -1 rbd::mirror::NamespaceReplayer: 0x28b4000 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707bd7b640 -1 rbd::mirror::NamespaceReplayer: 0x28b4e00 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f706ed61640 -1 rbd::mirror::MirrorStatusUpdater 0x476e360 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f706f562640 -1 rbd::mirror::MirrorStatusUpdater 0x44e90e0 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707bd7b640 -1 rbd::mirror::NamespaceReplayer: 0x28b56c0 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f706f562640 -1 rbd::mirror::MirrorStatusUpdater 0x476e240 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707c57c640 -1 rbd::mirror::NamespaceReplayer: 0x28b5340 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f7075d6f640 -1 rbd::mirror::MirrorStatusUpdater 0x44f2120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707c57c640 -1 rbd::mirror::NamespaceReplayer: 0x28b5500 handle_shut_down_remote_status_updater: error shutting remote mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707556e640 -1 rbd::mirror::MirrorStatusUpdater 0x476e120 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707c57c640 -1 rbd::mirror::NamespaceReplayer: 0x28b56c0 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f7075d6f640 -1 rbd::mirror::MirrorStatusUpdater 0x476e000 handle_shut_down_mirror_status_watcher: failed to shut down mirror status watcher: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707c57c640 -1 rbd::mirror::NamespaceReplayer: 0x28b5340 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707c57c640 -1 rbd::mirror::NamespaceReplayer: 0x28b5500 handle_shut_down_local_status_updater: error shutting local mirror status updater down: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.685+0000 7f707c57c640 -1 rbd::mirror::LeaderWatcher: 0x3cc3200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.trial080.stderr:2026-02-19T20:43:40.687+0000 7f707c57c640 -1 rbd::mirror::LeaderWatcher: 0x457d200 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (107) Transport endpoint is not connected 2026-02-19T20:43:40.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Stopped 2026-02-19T20:43:40.699 DEBUG:teuthology.run_tasks:Unwinding manager rbd-mirror 2026-02-19T20:43:40.718 DEBUG:tasks.rbd_mirror.cluster1.client.mirror.0:waiting for process to exit 2026-02-19T20:43:40.719 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:43:40.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.trial080.stderr:2026-02-19T20:43:40.737+0000 7f34c368e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 82230) UID: 1000 2026-02-19T20:43:40.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Stopped 2026-02-19T20:43:40.748 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-19T20:43:40.772 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2026-02-19T20:43:40.772 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2026-02-19T20:43:40.965 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:43:40.965 INFO:teuthology.orchestra.run.trial080.stderr:dumped all 2026-02-19T20:43:40.975 INFO:teuthology.orchestra.run.trial080.stdout:{"pg_ready":true,"pg_map":{"version":1461,"stamp":"2026-02-19T20:43:40.673575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3735016,"kb_used_data":9336,"kb_used_omap":2325,"kb_used_meta":3723306,"kb_avail":279380504,"statfs":{"total":289910292480,"available":286085636096,"internally_reserved":0,"allocated":9560064,"data_stored":6234186,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2381332,"internal_metadata":3812665836},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-536868918,"num_objects":-173,"num_object_clones":0,"num_object_copies":-346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-173,"num_whiteouts":0,"num_read":-112070,"num_read_kb":-39820630,"num_write":-1345889,"num_write_kb":-504303821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":-5,"num_objects_omap":-18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001166"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":768,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.655804+0000","last_change":"2026-02-19T20:07:50.516446+0000","last_active":"2026-02-19T20:43:08.655804+0000","last_peered":"2026-02-19T20:43:08.655804+0000","last_clean":"2026-02-19T20:43:08.655804+0000","last_became_active":"2026-02-19T20:07:48.515436+0000","last_became_peered":"2026-02-19T20:07:48.515436+0000","last_unstale":"2026-02-19T20:43:08.655804+0000","last_undegraded":"2026-02-19T20:43:08.655804+0000","last_fullsized":"2026-02-19T20:43:08.655804+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:20:02.908479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073110000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":768,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.655667+0000","last_change":"2026-02-19T20:07:50.516622+0000","last_active":"2026-02-19T20:43:08.655667+0000","last_peered":"2026-02-19T20:43:08.655667+0000","last_clean":"2026-02-19T20:43:08.655667+0000","last_became_active":"2026-02-19T20:07:48.515323+0000","last_became_peered":"2026-02-19T20:07:48.515323+0000","last_unstale":"2026-02-19T20:43:08.655667+0000","last_undegraded":"2026-02-19T20:43:08.655667+0000","last_fullsized":"2026-02-19T20:43:08.655667+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T07:26:46.646425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091905299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.656093+0000","last_change":"2026-02-19T20:07:50.516403+0000","last_active":"2026-02-19T20:43:08.656093+0000","last_peered":"2026-02-19T20:43:08.656093+0000","last_clean":"2026-02-19T20:43:08.656093+0000","last_became_active":"2026-02-19T20:07:48.515234+0000","last_became_peered":"2026-02-19T20:07:48.515234+0000","last_unstale":"2026-02-19T20:43:08.656093+0000","last_undegraded":"2026-02-19T20:43:08.656093+0000","last_fullsized":"2026-02-19T20:43:08.656093+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:29:42.841704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073914599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.655000+0000","last_change":"2026-02-19T20:07:50.516159+0000","last_active":"2026-02-19T20:43:08.655000+0000","last_peered":"2026-02-19T20:43:08.655000+0000","last_clean":"2026-02-19T20:43:08.655000+0000","last_became_active":"2026-02-19T20:07:48.515367+0000","last_became_peered":"2026-02-19T20:07:48.515367+0000","last_unstale":"2026-02-19T20:43:08.655000+0000","last_undegraded":"2026-02-19T20:43:08.655000+0000","last_fullsized":"2026-02-19T20:43:08.655000+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:50:02.278192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050451599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":772,"reported_epoch":383,"state":"active+clean","last_fresh":"2026-02-19T20:43:37.736162+0000","last_change":"2026-02-19T20:07:50.516971+0000","last_active":"2026-02-19T20:43:37.736162+0000","last_peered":"2026-02-19T20:43:37.736162+0000","last_clean":"2026-02-19T20:43:37.736162+0000","last_became_active":"2026-02-19T20:07:48.515048+0000","last_became_peered":"2026-02-19T20:07:48.515048+0000","last_unstale":"2026-02-19T20:43:37.736162+0000","last_undegraded":"2026-02-19T20:43:37.736162+0000","last_fullsized":"2026-02-19T20:43:37.736162+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:08:18.808510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590999999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654564+0000","last_change":"2026-02-19T20:07:50.559960+0000","last_active":"2026-02-19T20:43:08.654564+0000","last_peered":"2026-02-19T20:43:08.654564+0000","last_clean":"2026-02-19T20:43:08.654564+0000","last_became_active":"2026-02-19T20:07:48.514961+0000","last_became_peered":"2026-02-19T20:07:48.514961+0000","last_unstale":"2026-02-19T20:43:08.654564+0000","last_undegraded":"2026-02-19T20:43:08.654564+0000","last_fullsized":"2026-02-19T20:43:08.654564+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:28:50.223157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000202339,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654345+0000","last_change":"2026-02-19T20:07:50.559971+0000","last_active":"2026-02-19T20:43:08.654345+0000","last_peered":"2026-02-19T20:43:08.654345+0000","last_clean":"2026-02-19T20:43:08.654345+0000","last_became_active":"2026-02-19T20:07:48.515007+0000","last_became_peered":"2026-02-19T20:07:48.515007+0000","last_unstale":"2026-02-19T20:43:08.654345+0000","last_undegraded":"2026-02-19T20:43:08.654345+0000","last_fullsized":"2026-02-19T20:43:08.654345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:35:54.659935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000211387,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":770,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.656176+0000","last_change":"2026-02-19T20:07:50.516119+0000","last_active":"2026-02-19T20:43:08.656176+0000","last_peered":"2026-02-19T20:43:08.656176+0000","last_clean":"2026-02-19T20:43:08.656176+0000","last_became_active":"2026-02-19T20:07:48.514767+0000","last_became_peered":"2026-02-19T20:07:48.514767+0000","last_unstale":"2026-02-19T20:43:08.656176+0000","last_undegraded":"2026-02-19T20:43:08.656176+0000","last_fullsized":"2026-02-19T20:43:08.656176+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:16:18.619230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048350599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":860,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654370+0000","last_change":"2026-02-19T20:07:45.946467+0000","last_active":"2026-02-19T20:43:08.654370+0000","last_peered":"2026-02-19T20:43:08.654370+0000","last_clean":"2026-02-19T20:43:08.654370+0000","last_became_active":"2026-02-19T20:07:45.945977+0000","last_became_peered":"2026-02-19T20:07:45.945977+0000","last_unstale":"2026-02-19T20:43:08.654370+0000","last_undegraded":"2026-02-19T20:43:08.654370+0000","last_fullsized":"2026-02-19T20:43:08.654370+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_clean_scrub_stamp":"2026-02-19T20:07:44.502565+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:33.783466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738801,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":967120,"kb_used_data":3072,"kb_used_omap":753,"kb_used_meta":963278,"kb_avail":93404720,"statfs":{"total":96636764160,"available":95646433280,"internally_reserved":0,"allocated":3145728,"data_stored":1683120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":771731,"internal_metadata":986397037},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738801,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1202696,"kb_used_data":3128,"kb_used_omap":343,"kb_used_meta":1199208,"kb_avail":93169144,"statfs":{"total":96636764160,"available":95405203456,"internally_reserved":0,"allocated":3203072,"data_stored":2273488,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":351277,"internal_metadata":1227989971},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738802,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1565200,"kb_used_data":3136,"kb_used_omap":1228,"kb_used_meta":1560819,"kb_avail":92806640,"statfs":{"total":96636764160,"available":95033999360,"internally_reserved":0,"allocated":3211264,"data_stored":2277578,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1258324,"internal_metadata":1598278828},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:43:40.976 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2026-02-19T20:43:41.153 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:43:41.154 INFO:teuthology.orchestra.run.trial080.stderr:dumped all 2026-02-19T20:43:41.162 INFO:teuthology.orchestra.run.trial080.stdout:{"pg_ready":true,"pg_map":{"version":1461,"stamp":"2026-02-19T20:43:40.673575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3735016,"kb_used_data":9336,"kb_used_omap":2325,"kb_used_meta":3723306,"kb_avail":279380504,"statfs":{"total":289910292480,"available":286085636096,"internally_reserved":0,"allocated":9560064,"data_stored":6234186,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2381332,"internal_metadata":3812665836},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-536868918,"num_objects":-173,"num_object_clones":0,"num_object_copies":-346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-173,"num_whiteouts":0,"num_read":-112070,"num_read_kb":-39820630,"num_write":-1345889,"num_write_kb":-504303821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":-5,"num_objects_omap":-18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001166"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":768,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.655804+0000","last_change":"2026-02-19T20:07:50.516446+0000","last_active":"2026-02-19T20:43:08.655804+0000","last_peered":"2026-02-19T20:43:08.655804+0000","last_clean":"2026-02-19T20:43:08.655804+0000","last_became_active":"2026-02-19T20:07:48.515436+0000","last_became_peered":"2026-02-19T20:07:48.515436+0000","last_unstale":"2026-02-19T20:43:08.655804+0000","last_undegraded":"2026-02-19T20:43:08.655804+0000","last_fullsized":"2026-02-19T20:43:08.655804+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:20:02.908479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073110000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":768,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.655667+0000","last_change":"2026-02-19T20:07:50.516622+0000","last_active":"2026-02-19T20:43:08.655667+0000","last_peered":"2026-02-19T20:43:08.655667+0000","last_clean":"2026-02-19T20:43:08.655667+0000","last_became_active":"2026-02-19T20:07:48.515323+0000","last_became_peered":"2026-02-19T20:07:48.515323+0000","last_unstale":"2026-02-19T20:43:08.655667+0000","last_undegraded":"2026-02-19T20:43:08.655667+0000","last_fullsized":"2026-02-19T20:43:08.655667+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T07:26:46.646425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091905299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.656093+0000","last_change":"2026-02-19T20:07:50.516403+0000","last_active":"2026-02-19T20:43:08.656093+0000","last_peered":"2026-02-19T20:43:08.656093+0000","last_clean":"2026-02-19T20:43:08.656093+0000","last_became_active":"2026-02-19T20:07:48.515234+0000","last_became_peered":"2026-02-19T20:07:48.515234+0000","last_unstale":"2026-02-19T20:43:08.656093+0000","last_undegraded":"2026-02-19T20:43:08.656093+0000","last_fullsized":"2026-02-19T20:43:08.656093+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:29:42.841704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073914599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.655000+0000","last_change":"2026-02-19T20:07:50.516159+0000","last_active":"2026-02-19T20:43:08.655000+0000","last_peered":"2026-02-19T20:43:08.655000+0000","last_clean":"2026-02-19T20:43:08.655000+0000","last_became_active":"2026-02-19T20:07:48.515367+0000","last_became_peered":"2026-02-19T20:07:48.515367+0000","last_unstale":"2026-02-19T20:43:08.655000+0000","last_undegraded":"2026-02-19T20:43:08.655000+0000","last_fullsized":"2026-02-19T20:43:08.655000+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:50:02.278192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050451599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":772,"reported_epoch":383,"state":"active+clean","last_fresh":"2026-02-19T20:43:37.736162+0000","last_change":"2026-02-19T20:07:50.516971+0000","last_active":"2026-02-19T20:43:37.736162+0000","last_peered":"2026-02-19T20:43:37.736162+0000","last_clean":"2026-02-19T20:43:37.736162+0000","last_became_active":"2026-02-19T20:07:48.515048+0000","last_became_peered":"2026-02-19T20:07:48.515048+0000","last_unstale":"2026-02-19T20:43:37.736162+0000","last_undegraded":"2026-02-19T20:43:37.736162+0000","last_fullsized":"2026-02-19T20:43:37.736162+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:08:18.808510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590999999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654564+0000","last_change":"2026-02-19T20:07:50.559960+0000","last_active":"2026-02-19T20:43:08.654564+0000","last_peered":"2026-02-19T20:43:08.654564+0000","last_clean":"2026-02-19T20:43:08.654564+0000","last_became_active":"2026-02-19T20:07:48.514961+0000","last_became_peered":"2026-02-19T20:07:48.514961+0000","last_unstale":"2026-02-19T20:43:08.654564+0000","last_undegraded":"2026-02-19T20:43:08.654564+0000","last_fullsized":"2026-02-19T20:43:08.654564+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:28:50.223157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000202339,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654345+0000","last_change":"2026-02-19T20:07:50.559971+0000","last_active":"2026-02-19T20:43:08.654345+0000","last_peered":"2026-02-19T20:43:08.654345+0000","last_clean":"2026-02-19T20:43:08.654345+0000","last_became_active":"2026-02-19T20:07:48.515007+0000","last_became_peered":"2026-02-19T20:07:48.515007+0000","last_unstale":"2026-02-19T20:43:08.654345+0000","last_undegraded":"2026-02-19T20:43:08.654345+0000","last_fullsized":"2026-02-19T20:43:08.654345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:35:54.659935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000211387,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":770,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.656176+0000","last_change":"2026-02-19T20:07:50.516119+0000","last_active":"2026-02-19T20:43:08.656176+0000","last_peered":"2026-02-19T20:43:08.656176+0000","last_clean":"2026-02-19T20:43:08.656176+0000","last_became_active":"2026-02-19T20:07:48.514767+0000","last_became_peered":"2026-02-19T20:07:48.514767+0000","last_unstale":"2026-02-19T20:43:08.656176+0000","last_undegraded":"2026-02-19T20:43:08.656176+0000","last_fullsized":"2026-02-19T20:43:08.656176+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:16:18.619230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048350599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":860,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654370+0000","last_change":"2026-02-19T20:07:45.946467+0000","last_active":"2026-02-19T20:43:08.654370+0000","last_peered":"2026-02-19T20:43:08.654370+0000","last_clean":"2026-02-19T20:43:08.654370+0000","last_became_active":"2026-02-19T20:07:45.945977+0000","last_became_peered":"2026-02-19T20:07:45.945977+0000","last_unstale":"2026-02-19T20:43:08.654370+0000","last_undegraded":"2026-02-19T20:43:08.654370+0000","last_fullsized":"2026-02-19T20:43:08.654370+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_clean_scrub_stamp":"2026-02-19T20:07:44.502565+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:33.783466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738801,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":967120,"kb_used_data":3072,"kb_used_omap":753,"kb_used_meta":963278,"kb_avail":93404720,"statfs":{"total":96636764160,"available":95646433280,"internally_reserved":0,"allocated":3145728,"data_stored":1683120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":771731,"internal_metadata":986397037},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738801,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1202696,"kb_used_data":3128,"kb_used_omap":343,"kb_used_meta":1199208,"kb_avail":93169144,"statfs":{"total":96636764160,"available":95405203456,"internally_reserved":0,"allocated":3203072,"data_stored":2273488,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":351277,"internal_metadata":1227989971},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738802,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1565200,"kb_used_data":3136,"kb_used_omap":1228,"kb_used_meta":1560819,"kb_avail":92806640,"statfs":{"total":96636764160,"available":95033999360,"internally_reserved":0,"allocated":3211264,"data_stored":2277578,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1258324,"internal_metadata":1598278828},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:43:41.164 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2026-02-19T20:43:41.164 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2026-02-19T20:43:41.339 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:43:41.339 INFO:teuthology.orchestra.run.trial080.stderr:dumped all 2026-02-19T20:43:41.348 INFO:teuthology.orchestra.run.trial080.stdout:{"pg_ready":true,"pg_map":{"version":1461,"stamp":"2026-02-19T20:43:40.673575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3735016,"kb_used_data":9336,"kb_used_omap":2325,"kb_used_meta":3723306,"kb_avail":279380504,"statfs":{"total":289910292480,"available":286085636096,"internally_reserved":0,"allocated":9560064,"data_stored":6234186,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2381332,"internal_metadata":3812665836},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-536868918,"num_objects":-173,"num_object_clones":0,"num_object_copies":-346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-173,"num_whiteouts":0,"num_read":-112070,"num_read_kb":-39820630,"num_write":-1345889,"num_write_kb":-504303821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":-5,"num_objects_omap":-18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001166"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":768,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.655804+0000","last_change":"2026-02-19T20:07:50.516446+0000","last_active":"2026-02-19T20:43:08.655804+0000","last_peered":"2026-02-19T20:43:08.655804+0000","last_clean":"2026-02-19T20:43:08.655804+0000","last_became_active":"2026-02-19T20:07:48.515436+0000","last_became_peered":"2026-02-19T20:07:48.515436+0000","last_unstale":"2026-02-19T20:43:08.655804+0000","last_undegraded":"2026-02-19T20:43:08.655804+0000","last_fullsized":"2026-02-19T20:43:08.655804+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:20:02.908479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073110000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":768,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.655667+0000","last_change":"2026-02-19T20:07:50.516622+0000","last_active":"2026-02-19T20:43:08.655667+0000","last_peered":"2026-02-19T20:43:08.655667+0000","last_clean":"2026-02-19T20:43:08.655667+0000","last_became_active":"2026-02-19T20:07:48.515323+0000","last_became_peered":"2026-02-19T20:07:48.515323+0000","last_unstale":"2026-02-19T20:43:08.655667+0000","last_undegraded":"2026-02-19T20:43:08.655667+0000","last_fullsized":"2026-02-19T20:43:08.655667+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T07:26:46.646425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091905299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.656093+0000","last_change":"2026-02-19T20:07:50.516403+0000","last_active":"2026-02-19T20:43:08.656093+0000","last_peered":"2026-02-19T20:43:08.656093+0000","last_clean":"2026-02-19T20:43:08.656093+0000","last_became_active":"2026-02-19T20:07:48.515234+0000","last_became_peered":"2026-02-19T20:07:48.515234+0000","last_unstale":"2026-02-19T20:43:08.656093+0000","last_undegraded":"2026-02-19T20:43:08.656093+0000","last_fullsized":"2026-02-19T20:43:08.656093+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:29:42.841704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073914599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.655000+0000","last_change":"2026-02-19T20:07:50.516159+0000","last_active":"2026-02-19T20:43:08.655000+0000","last_peered":"2026-02-19T20:43:08.655000+0000","last_clean":"2026-02-19T20:43:08.655000+0000","last_became_active":"2026-02-19T20:07:48.515367+0000","last_became_peered":"2026-02-19T20:07:48.515367+0000","last_unstale":"2026-02-19T20:43:08.655000+0000","last_undegraded":"2026-02-19T20:43:08.655000+0000","last_fullsized":"2026-02-19T20:43:08.655000+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:50:02.278192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050451599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":772,"reported_epoch":383,"state":"active+clean","last_fresh":"2026-02-19T20:43:37.736162+0000","last_change":"2026-02-19T20:07:50.516971+0000","last_active":"2026-02-19T20:43:37.736162+0000","last_peered":"2026-02-19T20:43:37.736162+0000","last_clean":"2026-02-19T20:43:37.736162+0000","last_became_active":"2026-02-19T20:07:48.515048+0000","last_became_peered":"2026-02-19T20:07:48.515048+0000","last_unstale":"2026-02-19T20:43:37.736162+0000","last_undegraded":"2026-02-19T20:43:37.736162+0000","last_fullsized":"2026-02-19T20:43:37.736162+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:08:18.808510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590999999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654564+0000","last_change":"2026-02-19T20:07:50.559960+0000","last_active":"2026-02-19T20:43:08.654564+0000","last_peered":"2026-02-19T20:43:08.654564+0000","last_clean":"2026-02-19T20:43:08.654564+0000","last_became_active":"2026-02-19T20:07:48.514961+0000","last_became_peered":"2026-02-19T20:07:48.514961+0000","last_unstale":"2026-02-19T20:43:08.654564+0000","last_undegraded":"2026-02-19T20:43:08.654564+0000","last_fullsized":"2026-02-19T20:43:08.654564+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:28:50.223157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000202339,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654345+0000","last_change":"2026-02-19T20:07:50.559971+0000","last_active":"2026-02-19T20:43:08.654345+0000","last_peered":"2026-02-19T20:43:08.654345+0000","last_clean":"2026-02-19T20:43:08.654345+0000","last_became_active":"2026-02-19T20:07:48.515007+0000","last_became_peered":"2026-02-19T20:07:48.515007+0000","last_unstale":"2026-02-19T20:43:08.654345+0000","last_undegraded":"2026-02-19T20:43:08.654345+0000","last_fullsized":"2026-02-19T20:43:08.654345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:35:54.659935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000211387,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":770,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.656176+0000","last_change":"2026-02-19T20:07:50.516119+0000","last_active":"2026-02-19T20:43:08.656176+0000","last_peered":"2026-02-19T20:43:08.656176+0000","last_clean":"2026-02-19T20:43:08.656176+0000","last_became_active":"2026-02-19T20:07:48.514767+0000","last_became_peered":"2026-02-19T20:07:48.514767+0000","last_unstale":"2026-02-19T20:43:08.656176+0000","last_undegraded":"2026-02-19T20:43:08.656176+0000","last_fullsized":"2026-02-19T20:43:08.656176+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:16:18.619230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048350599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":860,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654370+0000","last_change":"2026-02-19T20:07:45.946467+0000","last_active":"2026-02-19T20:43:08.654370+0000","last_peered":"2026-02-19T20:43:08.654370+0000","last_clean":"2026-02-19T20:43:08.654370+0000","last_became_active":"2026-02-19T20:07:45.945977+0000","last_became_peered":"2026-02-19T20:07:45.945977+0000","last_unstale":"2026-02-19T20:43:08.654370+0000","last_undegraded":"2026-02-19T20:43:08.654370+0000","last_fullsized":"2026-02-19T20:43:08.654370+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_clean_scrub_stamp":"2026-02-19T20:07:44.502565+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:33.783466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738801,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":967120,"kb_used_data":3072,"kb_used_omap":753,"kb_used_meta":963278,"kb_avail":93404720,"statfs":{"total":96636764160,"available":95646433280,"internally_reserved":0,"allocated":3145728,"data_stored":1683120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":771731,"internal_metadata":986397037},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738801,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1202696,"kb_used_data":3128,"kb_used_omap":343,"kb_used_meta":1199208,"kb_avail":93169144,"statfs":{"total":96636764160,"available":95405203456,"internally_reserved":0,"allocated":3203072,"data_stored":2273488,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":351277,"internal_metadata":1227989971},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738802,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1565200,"kb_used_data":3136,"kb_used_omap":1228,"kb_used_meta":1560819,"kb_avail":92806640,"statfs":{"total":96636764160,"available":95033999360,"internally_reserved":0,"allocated":3211264,"data_stored":2277578,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1258324,"internal_metadata":1598278828},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:43:41.348 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2026-02-19T20:43:41.528 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:43:41.528 INFO:teuthology.orchestra.run.trial080.stdout:{"epoch":384,"fsid":"28c96ac4-2d59-44da-9699-cda71e572ddf","created":"2026-02-19T20:07:38.484071+0000","modified":"2026-02-19T20:43:39.728488+0000","last_up_change":"2026-02-19T20:07:42.491173+0000","last_in_change":"2026-02-19T20:07:39.160813+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-19T20:07:44.383359+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-19T20:07:46.578473+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":16,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e174f3ab-c860-44cb-b827-38110018fec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6800","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6801","nonce":515137397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6802","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6803","nonce":515137397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6806","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6807","nonce":515137397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6804","nonce":515137397},{"type":"v1","addr":"10.20.193.80:6805","nonce":515137397}]},"public_addr":"10.20.193.80:6801/515137397","cluster_addr":"10.20.193.80:6803/515137397","heartbeat_back_addr":"10.20.193.80:6807/515137397","heartbeat_front_addr":"10.20.193.80:6805/515137397","state":["exists","up"]},{"osd":1,"uuid":"6d926fb6-1591-422d-8ca1-58fb499c2772","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6808","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6810","nonce":2487769537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6812","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6814","nonce":2487769537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6820","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6822","nonce":2487769537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6816","nonce":2487769537},{"type":"v1","addr":"10.20.193.80:6818","nonce":2487769537}]},"public_addr":"10.20.193.80:6810/2487769537","cluster_addr":"10.20.193.80:6814/2487769537","heartbeat_back_addr":"10.20.193.80:6822/2487769537","heartbeat_front_addr":"10.20.193.80:6818/2487769537","state":["exists","up"]},{"osd":2,"uuid":"b259ad1f-166b-47e6-81f9-3e585f1bf53d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6809","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6811","nonce":4168087344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6813","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6815","nonce":4168087344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6821","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6823","nonce":4168087344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.80:6817","nonce":4168087344},{"type":"v1","addr":"10.20.193.80:6819","nonce":4168087344}]},"public_addr":"10.20.193.80:6811/4168087344","cluster_addr":"10.20.193.80:6815/4168087344","heartbeat_back_addr":"10.20.193.80:6823/4168087344","heartbeat_front_addr":"10.20.193.80:6819/4168087344","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.960240+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.940525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:40.959454+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:43:42.538 INFO:tasks.ceph:Scrubbing cluster2.osd.0 2026-02-19T20:43:42.538 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-19T20:43:42.620 INFO:teuthology.orchestra.run.trial080.stdout:{ 2026-02-19T20:43:42.620 INFO:teuthology.orchestra.run.trial080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T20:43:42.620 INFO:teuthology.orchestra.run.trial080.stdout:} 2026-02-19T20:43:42.625 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 0 2026-02-19T20:43:42.803 INFO:teuthology.orchestra.run.trial080.stderr:instructed osd(s) 0 to deep-scrub 2026-02-19T20:43:42.812 INFO:tasks.ceph:Scrubbing cluster2.osd.1 2026-02-19T20:43:42.813 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-19T20:43:42.894 INFO:teuthology.orchestra.run.trial080.stdout:{ 2026-02-19T20:43:42.894 INFO:teuthology.orchestra.run.trial080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-19T20:43:42.894 INFO:teuthology.orchestra.run.trial080.stdout:} 2026-02-19T20:43:42.900 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 1 2026-02-19T20:43:43.077 INFO:teuthology.orchestra.run.trial080.stderr:instructed osd(s) 1 to deep-scrub 2026-02-19T20:43:43.086 INFO:tasks.ceph:Scrubbing cluster2.osd.2 2026-02-19T20:43:43.086 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-19T20:43:43.165 INFO:teuthology.orchestra.run.trial080.stdout:{ 2026-02-19T20:43:43.165 INFO:teuthology.orchestra.run.trial080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-19T20:43:43.165 INFO:teuthology.orchestra.run.trial080.stdout:} 2026-02-19T20:43:43.170 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd deep-scrub 2 2026-02-19T20:43:43.347 INFO:teuthology.orchestra.run.trial080.stderr:instructed osd(s) 2 to deep-scrub 2026-02-19T20:43:43.356 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2026-02-19T20:43:43.530 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:43:43.530 INFO:teuthology.orchestra.run.trial080.stderr:dumped all 2026-02-19T20:43:43.539 INFO:teuthology.orchestra.run.trial080.stdout:{"pg_ready":true,"pg_map":{"version":1462,"stamp":"2026-02-19T20:43:42.673769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":232,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3753084,"kb_used_data":9356,"kb_used_omap":2327,"kb_used_meta":3741352,"kb_avail":279362436,"statfs":{"total":289910292480,"available":286067134464,"internally_reserved":0,"allocated":9580544,"data_stored":6241908,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2382948,"internal_metadata":3831145372},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-536868918,"num_objects":-173,"num_object_clones":0,"num_object_copies":-346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-173,"num_whiteouts":0,"num_read":-112078,"num_read_kb":-39820638,"num_write":-1345891,"num_write_kb":-504303823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":-5,"num_objects_omap":-18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001028"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":772,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:40.301229+0000","last_change":"2026-02-19T20:07:50.516446+0000","last_active":"2026-02-19T20:43:40.301229+0000","last_peered":"2026-02-19T20:43:40.301229+0000","last_clean":"2026-02-19T20:43:40.301229+0000","last_became_active":"2026-02-19T20:07:48.515436+0000","last_became_peered":"2026-02-19T20:07:48.515436+0000","last_unstale":"2026-02-19T20:43:40.301229+0000","last_undegraded":"2026-02-19T20:43:40.301229+0000","last_fullsized":"2026-02-19T20:43:40.301229+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:20:02.908479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073110000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":772,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:40.301237+0000","last_change":"2026-02-19T20:07:50.516622+0000","last_active":"2026-02-19T20:43:40.301237+0000","last_peered":"2026-02-19T20:43:40.301237+0000","last_clean":"2026-02-19T20:43:40.301237+0000","last_became_active":"2026-02-19T20:07:48.515323+0000","last_became_peered":"2026-02-19T20:07:48.515323+0000","last_unstale":"2026-02-19T20:43:40.301237+0000","last_undegraded":"2026-02-19T20:43:40.301237+0000","last_fullsized":"2026-02-19T20:43:40.301237+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T07:26:46.646425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091905299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":773,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:40.302026+0000","last_change":"2026-02-19T20:07:50.516403+0000","last_active":"2026-02-19T20:43:40.302026+0000","last_peered":"2026-02-19T20:43:40.302026+0000","last_clean":"2026-02-19T20:43:40.302026+0000","last_became_active":"2026-02-19T20:07:48.515234+0000","last_became_peered":"2026-02-19T20:07:48.515234+0000","last_unstale":"2026-02-19T20:43:40.302026+0000","last_undegraded":"2026-02-19T20:43:40.302026+0000","last_fullsized":"2026-02-19T20:43:40.302026+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:29:42.841704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073914599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":773,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:40.301390+0000","last_change":"2026-02-19T20:07:50.516159+0000","last_active":"2026-02-19T20:43:40.301390+0000","last_peered":"2026-02-19T20:43:40.301390+0000","last_clean":"2026-02-19T20:43:40.301390+0000","last_became_active":"2026-02-19T20:07:48.515367+0000","last_became_peered":"2026-02-19T20:07:48.515367+0000","last_unstale":"2026-02-19T20:43:40.301390+0000","last_undegraded":"2026-02-19T20:43:40.301390+0000","last_fullsized":"2026-02-19T20:43:40.301390+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:50:02.278192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050451599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'2","reported_seq":772,"reported_epoch":383,"state":"active+clean","last_fresh":"2026-02-19T20:43:37.736162+0000","last_change":"2026-02-19T20:07:50.516971+0000","last_active":"2026-02-19T20:43:37.736162+0000","last_peered":"2026-02-19T20:43:37.736162+0000","last_clean":"2026-02-19T20:43:37.736162+0000","last_became_active":"2026-02-19T20:07:48.515048+0000","last_became_peered":"2026-02-19T20:07:48.515048+0000","last_unstale":"2026-02-19T20:43:37.736162+0000","last_undegraded":"2026-02-19T20:43:37.736162+0000","last_fullsized":"2026-02-19T20:43:37.736162+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:08:18.808510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097590999999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654564+0000","last_change":"2026-02-19T20:07:50.559960+0000","last_active":"2026-02-19T20:43:08.654564+0000","last_peered":"2026-02-19T20:43:08.654564+0000","last_clean":"2026-02-19T20:43:08.654564+0000","last_became_active":"2026-02-19T20:07:48.514961+0000","last_became_peered":"2026-02-19T20:07:48.514961+0000","last_unstale":"2026-02-19T20:43:08.654564+0000","last_undegraded":"2026-02-19T20:43:08.654564+0000","last_fullsized":"2026-02-19T20:43:08.654564+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:28:50.223157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000202339,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":769,"reported_epoch":382,"state":"active+clean","last_fresh":"2026-02-19T20:43:08.654345+0000","last_change":"2026-02-19T20:07:50.559971+0000","last_active":"2026-02-19T20:43:08.654345+0000","last_peered":"2026-02-19T20:43:08.654345+0000","last_clean":"2026-02-19T20:43:08.654345+0000","last_became_active":"2026-02-19T20:07:48.515007+0000","last_became_peered":"2026-02-19T20:07:48.515007+0000","last_unstale":"2026-02-19T20:43:08.654345+0000","last_undegraded":"2026-02-19T20:43:08.654345+0000","last_fullsized":"2026-02-19T20:43:08.654345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:35:54.659935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000211387,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"14'1","reported_seq":774,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:40.301420+0000","last_change":"2026-02-19T20:07:50.516119+0000","last_active":"2026-02-19T20:43:40.301420+0000","last_peered":"2026-02-19T20:43:40.301420+0000","last_clean":"2026-02-19T20:43:40.301420+0000","last_became_active":"2026-02-19T20:07:48.514767+0000","last_became_peered":"2026-02-19T20:07:48.514767+0000","last_unstale":"2026-02-19T20:43:40.301420+0000","last_undegraded":"2026-02-19T20:43:40.301420+0000","last_fullsized":"2026-02-19T20:43:40.301420+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:47.507510+0000","last_clean_scrub_stamp":"2026-02-19T20:07:47.507510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:16:18.619230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048350599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":864,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:40.301973+0000","last_change":"2026-02-19T20:07:45.946467+0000","last_active":"2026-02-19T20:43:40.301973+0000","last_peered":"2026-02-19T20:43:40.301973+0000","last_clean":"2026-02-19T20:43:40.301973+0000","last_became_active":"2026-02-19T20:07:45.945977+0000","last_became_peered":"2026-02-19T20:07:45.945977+0000","last_unstale":"2026-02-19T20:43:40.301973+0000","last_undegraded":"2026-02-19T20:43:40.301973+0000","last_fullsized":"2026-02-19T20:43:40.301973+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:44.502565+0000","last_clean_scrub_stamp":"2026-02-19T20:07:44.502565+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:33.783466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738801,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":967120,"kb_used_data":3072,"kb_used_omap":753,"kb_used_meta":963278,"kb_avail":93404720,"statfs":{"total":96636764160,"available":95646433280,"internally_reserved":0,"allocated":3145728,"data_stored":1683120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":771731,"internal_metadata":986397037},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738802,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1220764,"kb_used_data":3148,"kb_used_omap":344,"kb_used_meta":1217255,"kb_avail":93151076,"statfs":{"total":96636764160,"available":95386701824,"internally_reserved":0,"allocated":3223552,"data_stored":2281210,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":352893,"internal_metadata":1246469507},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738802,"num_pgs":96,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1565200,"kb_used_data":3136,"kb_used_omap":1228,"kb_used_meta":1560819,"kb_avail":92806640,"statfs":{"total":96636764160,"available":95033999360,"internally_reserved":0,"allocated":3211264,"data_stored":2277578,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1258324,"internal_metadata":1598278828},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:43:43.542 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-19T20:07:47.507510+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=47, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=43, tm_sec=41, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:43:43.542 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-19T20:07:47.507510+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=47, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=43, tm_sec=41, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:43:43.543 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-19T20:07:47.507510+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=47, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=43, tm_sec=41, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:43:43.543 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-19T20:07:47.507510+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=47, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=43, tm_sec=41, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:43:43.543 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-19T20:07:47.507510+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=47, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=43, tm_sec=41, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:43:43.543 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-19T20:07:47.507510+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=47, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=43, tm_sec=41, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:43:43.543 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-19T20:07:47.507510+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=47, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=43, tm_sec=41, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:43:43.543 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-19T20:07:47.507510+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=47, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=43, tm_sec=41, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:43:43.543 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-19T20:07:44.502565+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=44, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=43, tm_sec=41, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:43:43.543 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-19T20:44:03.545 DEBUG:teuthology.orchestra.run.trial080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2026-02-19T20:44:03.723 INFO:teuthology.orchestra.run.trial080.stdout: 2026-02-19T20:44:03.723 INFO:teuthology.orchestra.run.trial080.stderr:dumped all 2026-02-19T20:44:03.731 INFO:teuthology.orchestra.run.trial080.stdout:{"pg_ready":true,"pg_map":{"version":1472,"stamp":"2026-02-19T20:44:02.675348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":3771096,"kb_used_data":9384,"kb_used_omap":2331,"kb_used_meta":3759332,"kb_avail":279344424,"statfs":{"total":289910292480,"available":286048690176,"internally_reserved":0,"allocated":9609216,"data_stored":6253243,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2387548,"internal_metadata":3849556388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000914"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":780,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:48.825637+0000","last_change":"2026-02-19T20:43:48.825637+0000","last_active":"2026-02-19T20:43:48.825637+0000","last_peered":"2026-02-19T20:43:48.825637+0000","last_clean":"2026-02-19T20:43:48.825637+0000","last_became_active":"2026-02-19T20:07:48.515436+0000","last_became_peered":"2026-02-19T20:07:48.515436+0000","last_unstale":"2026-02-19T20:43:48.825637+0000","last_undegraded":"2026-02-19T20:43:48.825637+0000","last_fullsized":"2026-02-19T20:43:48.825637+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:43:48.825593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:43:48.825593+0000","last_clean_scrub_stamp":"2026-02-19T20:43:48.825593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T21:30:54.506506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073110000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":780,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:46.780937+0000","last_change":"2026-02-19T20:43:46.780937+0000","last_active":"2026-02-19T20:43:46.780937+0000","last_peered":"2026-02-19T20:43:46.780937+0000","last_clean":"2026-02-19T20:43:46.780937+0000","last_became_active":"2026-02-19T20:07:48.515323+0000","last_became_peered":"2026-02-19T20:07:48.515323+0000","last_unstale":"2026-02-19T20:43:46.780937+0000","last_undegraded":"2026-02-19T20:43:46.780937+0000","last_fullsized":"2026-02-19T20:43:46.780937+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:43:46.780899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:43:46.780899+0000","last_clean_scrub_stamp":"2026-02-19T20:43:46.780899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:01:35.881454+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00091905299999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":781,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:44.797484+0000","last_change":"2026-02-19T20:43:44.797484+0000","last_active":"2026-02-19T20:43:44.797484+0000","last_peered":"2026-02-19T20:43:44.797484+0000","last_clean":"2026-02-19T20:43:44.797484+0000","last_became_active":"2026-02-19T20:07:48.515234+0000","last_became_peered":"2026-02-19T20:07:48.515234+0000","last_unstale":"2026-02-19T20:43:44.797484+0000","last_undegraded":"2026-02-19T20:43:44.797484+0000","last_fullsized":"2026-02-19T20:43:44.797484+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:43:44.797446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:43:44.797446+0000","last_clean_scrub_stamp":"2026-02-19T20:43:44.797446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:58:01.503880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073914599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":781,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:45.758273+0000","last_change":"2026-02-19T20:43:45.758273+0000","last_active":"2026-02-19T20:43:45.758273+0000","last_peered":"2026-02-19T20:43:45.758273+0000","last_clean":"2026-02-19T20:43:45.758273+0000","last_became_active":"2026-02-19T20:07:48.515367+0000","last_became_peered":"2026-02-19T20:07:48.515367+0000","last_unstale":"2026-02-19T20:43:45.758273+0000","last_undegraded":"2026-02-19T20:43:45.758273+0000","last_fullsized":"2026-02-19T20:43:45.758273+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:43:45.758159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:43:45.758159+0000","last_clean_scrub_stamp":"2026-02-19T20:43:45.758159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:31:02.511952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050451599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"384'3","reported_seq":783,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:43.079209+0000","last_change":"2026-02-19T20:43:43.079209+0000","last_active":"2026-02-19T20:43:43.079209+0000","last_peered":"2026-02-19T20:43:43.079209+0000","last_clean":"2026-02-19T20:43:43.079209+0000","last_became_active":"2026-02-19T20:07:48.515048+0000","last_became_peered":"2026-02-19T20:07:48.515048+0000","last_unstale":"2026-02-19T20:43:43.079209+0000","last_undegraded":"2026-02-19T20:43:43.079209+0000","last_fullsized":"2026-02-19T20:43:43.079209+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"384'3","last_scrub_stamp":"2026-02-19T20:43:43.079169+0000","last_deep_scrub":"384'3","last_deep_scrub_stamp":"2026-02-19T20:43:43.079169+0000","last_clean_scrub_stamp":"2026-02-19T20:43:43.079169+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:16:30.934360+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00097590999999999997,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":781,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:44.995884+0000","last_change":"2026-02-19T20:43:44.995884+0000","last_active":"2026-02-19T20:43:44.995884+0000","last_peered":"2026-02-19T20:43:44.995884+0000","last_clean":"2026-02-19T20:43:44.995884+0000","last_became_active":"2026-02-19T20:07:48.514961+0000","last_became_peered":"2026-02-19T20:07:48.514961+0000","last_unstale":"2026-02-19T20:43:44.995884+0000","last_undegraded":"2026-02-19T20:43:44.995884+0000","last_fullsized":"2026-02-19T20:43:44.995884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:43:44.995806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:43:44.995806+0000","last_clean_scrub_stamp":"2026-02-19T20:43:44.995806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:41:44.410720+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000202339,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":781,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:44.004613+0000","last_change":"2026-02-19T20:43:44.004613+0000","last_active":"2026-02-19T20:43:44.004613+0000","last_peered":"2026-02-19T20:43:44.004613+0000","last_clean":"2026-02-19T20:43:44.004613+0000","last_became_active":"2026-02-19T20:07:48.515007+0000","last_became_peered":"2026-02-19T20:07:48.515007+0000","last_unstale":"2026-02-19T20:43:44.004613+0000","last_undegraded":"2026-02-19T20:43:44.004613+0000","last_fullsized":"2026-02-19T20:43:44.004613+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:43:44.004511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:43:44.004511+0000","last_clean_scrub_stamp":"2026-02-19T20:43:44.004511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T04:44:59.588880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000211387,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"384'2","reported_seq":783,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:43.819488+0000","last_change":"2026-02-19T20:43:43.819488+0000","last_active":"2026-02-19T20:43:43.819488+0000","last_peered":"2026-02-19T20:43:43.819488+0000","last_clean":"2026-02-19T20:43:43.819488+0000","last_became_active":"2026-02-19T20:07:48.514767+0000","last_became_peered":"2026-02-19T20:07:48.514767+0000","last_unstale":"2026-02-19T20:43:43.819488+0000","last_undegraded":"2026-02-19T20:43:43.819488+0000","last_fullsized":"2026-02-19T20:43:43.819488+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"384'2","last_scrub_stamp":"2026-02-19T20:43:43.819446+0000","last_deep_scrub":"384'2","last_deep_scrub_stamp":"2026-02-19T20:43:43.819446+0000","last_clean_scrub_stamp":"2026-02-19T20:43:43.819446+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:11:12.805274+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00048350599999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"384'74","reported_seq":875,"reported_epoch":384,"state":"active+clean","last_fresh":"2026-02-19T20:43:47.829246+0000","last_change":"2026-02-19T20:43:47.829224+0000","last_active":"2026-02-19T20:43:47.829246+0000","last_peered":"2026-02-19T20:43:47.829246+0000","last_clean":"2026-02-19T20:43:47.829246+0000","last_became_active":"2026-02-19T20:07:45.945977+0000","last_became_peered":"2026-02-19T20:07:45.945977+0000","last_unstale":"2026-02-19T20:43:47.829246+0000","last_undegraded":"2026-02-19T20:43:47.829246+0000","last_fullsized":"2026-02-19T20:43:47.829246+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"384'74","last_scrub_stamp":"2026-02-19T20:43:47.829186+0000","last_deep_scrub":"384'74","last_deep_scrub_stamp":"2026-02-19T20:43:47.829186+0000","last_clean_scrub_stamp":"2026-02-19T20:43:47.829186+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T05:10:17.306410+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12990,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738805,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":985120,"kb_used_data":3088,"kb_used_omap":755,"kb_used_meta":981260,"kb_avail":93386720,"statfs":{"total":96636764160,"available":95628001280,"internally_reserved":0,"allocated":3162112,"data_stored":1690823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":773735,"internal_metadata":1004810649},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738806,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1220764,"kb_used_data":3148,"kb_used_omap":344,"kb_used_meta":1217255,"kb_avail":93151076,"statfs":{"total":96636764160,"available":95386701824,"internally_reserved":0,"allocated":3223552,"data_stored":2281210,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":352893,"internal_metadata":1246469507},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738806,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1565212,"kb_used_data":3148,"kb_used_omap":1231,"kb_used_meta":1560816,"kb_avail":92806628,"statfs":{"total":96636764160,"available":95033987072,"internally_reserved":0,"allocated":3223552,"data_stored":2281210,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1260920,"internal_metadata":1598276232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6495,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:44:03.732 DEBUG:teuthology.orchestra.run.trial080:> sudo ceph --cluster cluster2 config set global mon_health_to_clog false 2026-02-19T20:44:03.919 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-19T20:44:03.919 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-19T20:44:03.919 DEBUG:tasks.ceph.cluster2.osd.0:waiting for process to exit 2026-02-19T20:44:03.920 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:03.993 INFO:tasks.ceph.cluster2.osd.0:Stopped 2026-02-19T20:44:03.993 DEBUG:tasks.ceph.cluster2.osd.1:waiting for process to exit 2026-02-19T20:44:03.993 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:04.060 INFO:tasks.ceph.cluster2.osd.1:Stopped 2026-02-19T20:44:04.060 DEBUG:tasks.ceph.cluster2.osd.2:waiting for process to exit 2026-02-19T20:44:04.060 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:04.129 INFO:tasks.ceph.cluster2.osd.2:Stopped 2026-02-19T20:44:04.129 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-19T20:44:04.129 DEBUG:tasks.ceph.cluster2.mgr.x:waiting for process to exit 2026-02-19T20:44:04.129 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:04.174 INFO:tasks.ceph.cluster2.mgr.x:Stopped 2026-02-19T20:44:04.174 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-19T20:44:04.174 DEBUG:tasks.ceph.cluster2.mon.a:waiting for process to exit 2026-02-19T20:44:04.174 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:04.185 INFO:tasks.ceph.cluster2.mon.a:Stopped 2026-02-19T20:44:04.185 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-19T20:44:04.185 DEBUG:teuthology.orchestra.run.trial080:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cluster2.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(OSD_SLOW_PING_TIME' | head -n 1 2026-02-19T20:44:04.250 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-0 on ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:44:04.250 DEBUG:teuthology.orchestra.run.trial080:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-0 2026-02-19T20:44:04.506 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-1 on ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:44:04.506 DEBUG:teuthology.orchestra.run.trial080:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-1 2026-02-19T20:44:04.697 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster2-2 on ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:44:04.697 DEBUG:teuthology.orchestra.run.trial080:> sync && sudo umount -f /var/lib/ceph/osd/cluster2-2 2026-02-19T20:44:04.890 INFO:tasks.ceph:Archiving mon data... 2026-02-19T20:44:04.892 DEBUG:teuthology.misc:Transferring archived files from trial080:/var/lib/ceph/mon/cluster2-a to /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606/data/cluster2.mon.a.tgz 2026-02-19T20:44:04.892 DEBUG:teuthology.orchestra.run.trial080:> mktemp 2026-02-19T20:44:04.908 INFO:teuthology.orchestra.run.trial080.stdout:/tmp/tmp.PYc3HyGeUm 2026-02-19T20:44:04.908 DEBUG:teuthology.orchestra.run.trial080:> sudo tar cz -f - -C /var/lib/ceph/mon/cluster2-a -- . > /tmp/tmp.PYc3HyGeUm 2026-02-19T20:44:05.018 DEBUG:teuthology.orchestra.run.trial080:> sudo chmod 0666 /tmp/tmp.PYc3HyGeUm 2026-02-19T20:44:05.100 DEBUG:teuthology.orchestra.remote:trial080:/tmp/tmp.PYc3HyGeUm is 367KB 2026-02-19T20:44:05.166 DEBUG:teuthology.orchestra.run.trial080:> rm -fr /tmp/tmp.PYc3HyGeUm 2026-02-19T20:44:05.184 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-19T20:44:05.184 DEBUG:teuthology.orchestra.run.trial073:> sudo rm -rf -- /etc/ceph/cluster2.conf /etc/ceph/cluster2.keyring /home/ubuntu/cephtest/cluster2.data /home/ubuntu/cephtest/cluster2.monmap /home/ubuntu/cephtest/../*.pid 2026-02-19T20:44:05.187 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -rf -- /etc/ceph/cluster2.conf /etc/ceph/cluster2.keyring /home/ubuntu/cephtest/cluster2.data /home/ubuntu/cephtest/cluster2.monmap /home/ubuntu/cephtest/../*.pid 2026-02-19T20:44:05.249 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-19T20:44:05.272 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2026-02-19T20:44:05.273 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2026-02-19T20:44:05.450 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:44:05.451 INFO:teuthology.orchestra.run.trial073.stderr:dumped all 2026-02-19T20:44:05.459 INFO:teuthology.orchestra.run.trial073.stdout:{"pg_ready":true,"pg_map":{"version":1553,"stamp":"2026-02-19T20:44:05.157968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":200232,"kb_used_data":11064,"kb_used_omap":7959,"kb_used_meta":181160,"kb_avail":282915288,"statfs":{"total":289910292480,"available":289705254912,"internally_reserved":0,"allocated":11329536,"data_stored":7191613,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":8150047,"internal_metadata":185508833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000974"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":914,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.277460+0000","last_change":"2026-02-19T20:07:27.870930+0000","last_active":"2026-02-19T20:43:39.277460+0000","last_peered":"2026-02-19T20:43:39.277460+0000","last_clean":"2026-02-19T20:43:39.277460+0000","last_became_active":"2026-02-19T20:07:25.866966+0000","last_became_peered":"2026-02-19T20:07:25.866966+0000","last_unstale":"2026-02-19T20:43:39.277460+0000","last_undegraded":"2026-02-19T20:43:39.277460+0000","last_fullsized":"2026-02-19T20:43:39.277460+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:08:33.157778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053370000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":914,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.278148+0000","last_change":"2026-02-19T20:07:27.871242+0000","last_active":"2026-02-19T20:43:39.278148+0000","last_peered":"2026-02-19T20:43:39.278148+0000","last_clean":"2026-02-19T20:43:39.278148+0000","last_became_active":"2026-02-19T20:07:25.867278+0000","last_became_peered":"2026-02-19T20:07:25.867278+0000","last_unstale":"2026-02-19T20:43:39.278148+0000","last_undegraded":"2026-02-19T20:43:39.278148+0000","last_fullsized":"2026-02-19T20:43:39.278148+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T05:18:57.062974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081141899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.278308+0000","last_change":"2026-02-19T20:07:27.871411+0000","last_active":"2026-02-19T20:43:39.278308+0000","last_peered":"2026-02-19T20:43:39.278308+0000","last_clean":"2026-02-19T20:43:39.278308+0000","last_became_active":"2026-02-19T20:07:25.867250+0000","last_became_peered":"2026-02-19T20:07:25.867250+0000","last_unstale":"2026-02-19T20:43:39.278308+0000","last_undegraded":"2026-02-19T20:43:39.278308+0000","last_fullsized":"2026-02-19T20:43:39.278308+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:55:29.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094229399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.275854+0000","last_change":"2026-02-19T20:07:27.871199+0000","last_active":"2026-02-19T20:43:39.275854+0000","last_peered":"2026-02-19T20:43:39.275854+0000","last_clean":"2026-02-19T20:43:39.275854+0000","last_became_active":"2026-02-19T20:07:25.867041+0000","last_became_peered":"2026-02-19T20:07:25.867041+0000","last_unstale":"2026-02-19T20:43:39.275854+0000","last_undegraded":"2026-02-19T20:43:39.275854+0000","last_fullsized":"2026-02-19T20:43:39.275854+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:09:11.855713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079016000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:38.593644+0000","last_change":"2026-02-19T20:07:27.869613+0000","last_active":"2026-02-19T20:43:38.593644+0000","last_peered":"2026-02-19T20:43:38.593644+0000","last_clean":"2026-02-19T20:43:38.593644+0000","last_became_active":"2026-02-19T20:07:25.866844+0000","last_became_peered":"2026-02-19T20:07:25.866844+0000","last_unstale":"2026-02-19T20:43:38.593644+0000","last_undegraded":"2026-02-19T20:43:38.593644+0000","last_fullsized":"2026-02-19T20:43:38.593644+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:23.866049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055244400000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.825177+0000","last_change":"2026-02-19T20:07:27.986078+0000","last_active":"2026-02-19T20:43:39.825177+0000","last_peered":"2026-02-19T20:43:39.825177+0000","last_clean":"2026-02-19T20:43:39.825177+0000","last_became_active":"2026-02-19T20:07:25.867756+0000","last_became_peered":"2026-02-19T20:07:25.867756+0000","last_unstale":"2026-02-19T20:43:39.825177+0000","last_undegraded":"2026-02-19T20:43:39.825177+0000","last_fullsized":"2026-02-19T20:43:39.825177+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:22:24.470302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000248385,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.823145+0000","last_change":"2026-02-19T20:07:27.986095+0000","last_active":"2026-02-19T20:43:39.823145+0000","last_peered":"2026-02-19T20:43:39.823145+0000","last_clean":"2026-02-19T20:43:39.823145+0000","last_became_active":"2026-02-19T20:07:25.867520+0000","last_became_peered":"2026-02-19T20:07:25.867520+0000","last_unstale":"2026-02-19T20:43:39.823145+0000","last_undegraded":"2026-02-19T20:43:39.823145+0000","last_fullsized":"2026-02-19T20:43:39.823145+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:16:42.961584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026667900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.277825+0000","last_change":"2026-02-19T20:07:27.870999+0000","last_active":"2026-02-19T20:43:39.277825+0000","last_peered":"2026-02-19T20:43:39.277825+0000","last_clean":"2026-02-19T20:43:39.277825+0000","last_became_active":"2026-02-19T20:07:25.866443+0000","last_became_peered":"2026-02-19T20:07:25.866443+0000","last_unstale":"2026-02-19T20:43:39.277825+0000","last_undegraded":"2026-02-19T20:43:39.277825+0000","last_fullsized":"2026-02-19T20:43:39.277825+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:57:49.025933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059055499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'72","reported_seq":1007,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.274899+0000","last_change":"2026-02-19T20:07:23.081147+0000","last_active":"2026-02-19T20:43:39.274899+0000","last_peered":"2026-02-19T20:43:39.274899+0000","last_clean":"2026-02-19T20:43:39.274899+0000","last_became_active":"2026-02-19T20:07:23.079045+0000","last_became_peered":"2026-02-19T20:07:23.079045+0000","last_unstale":"2026-02-19T20:43:39.274899+0000","last_undegraded":"2026-02-19T20:43:39.274899+0000","last_fullsized":"2026-02-19T20:43:39.274899+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_clean_scrub_stamp":"2026-02-19T20:07:21.856480+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T01:22:07.001571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738812,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66704,"kb_used_data":3648,"kb_used_omap":2700,"kb_used_meta":60339,"kb_avail":94305136,"statfs":{"total":96636764160,"available":96568459264,"internally_reserved":0,"allocated":3735552,"data_stored":2003613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2765235,"internal_metadata":61787725},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738813,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":2175,"kb_used_meta":60864,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":2227362,"internal_metadata":62325598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738810,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":3083,"kb_used_meta":59956,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":3157450,"internal_metadata":61395510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:44:05.459 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2026-02-19T20:44:05.636 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:44:05.636 INFO:teuthology.orchestra.run.trial073.stderr:dumped all 2026-02-19T20:44:05.644 INFO:teuthology.orchestra.run.trial073.stdout:{"pg_ready":true,"pg_map":{"version":1553,"stamp":"2026-02-19T20:44:05.157968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":200232,"kb_used_data":11064,"kb_used_omap":7959,"kb_used_meta":181160,"kb_avail":282915288,"statfs":{"total":289910292480,"available":289705254912,"internally_reserved":0,"allocated":11329536,"data_stored":7191613,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":8150047,"internal_metadata":185508833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000974"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":914,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.277460+0000","last_change":"2026-02-19T20:07:27.870930+0000","last_active":"2026-02-19T20:43:39.277460+0000","last_peered":"2026-02-19T20:43:39.277460+0000","last_clean":"2026-02-19T20:43:39.277460+0000","last_became_active":"2026-02-19T20:07:25.866966+0000","last_became_peered":"2026-02-19T20:07:25.866966+0000","last_unstale":"2026-02-19T20:43:39.277460+0000","last_undegraded":"2026-02-19T20:43:39.277460+0000","last_fullsized":"2026-02-19T20:43:39.277460+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:08:33.157778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053370000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":914,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.278148+0000","last_change":"2026-02-19T20:07:27.871242+0000","last_active":"2026-02-19T20:43:39.278148+0000","last_peered":"2026-02-19T20:43:39.278148+0000","last_clean":"2026-02-19T20:43:39.278148+0000","last_became_active":"2026-02-19T20:07:25.867278+0000","last_became_peered":"2026-02-19T20:07:25.867278+0000","last_unstale":"2026-02-19T20:43:39.278148+0000","last_undegraded":"2026-02-19T20:43:39.278148+0000","last_fullsized":"2026-02-19T20:43:39.278148+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T05:18:57.062974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081141899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.278308+0000","last_change":"2026-02-19T20:07:27.871411+0000","last_active":"2026-02-19T20:43:39.278308+0000","last_peered":"2026-02-19T20:43:39.278308+0000","last_clean":"2026-02-19T20:43:39.278308+0000","last_became_active":"2026-02-19T20:07:25.867250+0000","last_became_peered":"2026-02-19T20:07:25.867250+0000","last_unstale":"2026-02-19T20:43:39.278308+0000","last_undegraded":"2026-02-19T20:43:39.278308+0000","last_fullsized":"2026-02-19T20:43:39.278308+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:55:29.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094229399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.275854+0000","last_change":"2026-02-19T20:07:27.871199+0000","last_active":"2026-02-19T20:43:39.275854+0000","last_peered":"2026-02-19T20:43:39.275854+0000","last_clean":"2026-02-19T20:43:39.275854+0000","last_became_active":"2026-02-19T20:07:25.867041+0000","last_became_peered":"2026-02-19T20:07:25.867041+0000","last_unstale":"2026-02-19T20:43:39.275854+0000","last_undegraded":"2026-02-19T20:43:39.275854+0000","last_fullsized":"2026-02-19T20:43:39.275854+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:09:11.855713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079016000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:38.593644+0000","last_change":"2026-02-19T20:07:27.869613+0000","last_active":"2026-02-19T20:43:38.593644+0000","last_peered":"2026-02-19T20:43:38.593644+0000","last_clean":"2026-02-19T20:43:38.593644+0000","last_became_active":"2026-02-19T20:07:25.866844+0000","last_became_peered":"2026-02-19T20:07:25.866844+0000","last_unstale":"2026-02-19T20:43:38.593644+0000","last_undegraded":"2026-02-19T20:43:38.593644+0000","last_fullsized":"2026-02-19T20:43:38.593644+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:23.866049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055244400000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.825177+0000","last_change":"2026-02-19T20:07:27.986078+0000","last_active":"2026-02-19T20:43:39.825177+0000","last_peered":"2026-02-19T20:43:39.825177+0000","last_clean":"2026-02-19T20:43:39.825177+0000","last_became_active":"2026-02-19T20:07:25.867756+0000","last_became_peered":"2026-02-19T20:07:25.867756+0000","last_unstale":"2026-02-19T20:43:39.825177+0000","last_undegraded":"2026-02-19T20:43:39.825177+0000","last_fullsized":"2026-02-19T20:43:39.825177+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:22:24.470302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000248385,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.823145+0000","last_change":"2026-02-19T20:07:27.986095+0000","last_active":"2026-02-19T20:43:39.823145+0000","last_peered":"2026-02-19T20:43:39.823145+0000","last_clean":"2026-02-19T20:43:39.823145+0000","last_became_active":"2026-02-19T20:07:25.867520+0000","last_became_peered":"2026-02-19T20:07:25.867520+0000","last_unstale":"2026-02-19T20:43:39.823145+0000","last_undegraded":"2026-02-19T20:43:39.823145+0000","last_fullsized":"2026-02-19T20:43:39.823145+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:16:42.961584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026667900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.277825+0000","last_change":"2026-02-19T20:07:27.870999+0000","last_active":"2026-02-19T20:43:39.277825+0000","last_peered":"2026-02-19T20:43:39.277825+0000","last_clean":"2026-02-19T20:43:39.277825+0000","last_became_active":"2026-02-19T20:07:25.866443+0000","last_became_peered":"2026-02-19T20:07:25.866443+0000","last_unstale":"2026-02-19T20:43:39.277825+0000","last_undegraded":"2026-02-19T20:43:39.277825+0000","last_fullsized":"2026-02-19T20:43:39.277825+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:57:49.025933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059055499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'72","reported_seq":1007,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.274899+0000","last_change":"2026-02-19T20:07:23.081147+0000","last_active":"2026-02-19T20:43:39.274899+0000","last_peered":"2026-02-19T20:43:39.274899+0000","last_clean":"2026-02-19T20:43:39.274899+0000","last_became_active":"2026-02-19T20:07:23.079045+0000","last_became_peered":"2026-02-19T20:07:23.079045+0000","last_unstale":"2026-02-19T20:43:39.274899+0000","last_undegraded":"2026-02-19T20:43:39.274899+0000","last_fullsized":"2026-02-19T20:43:39.274899+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_clean_scrub_stamp":"2026-02-19T20:07:21.856480+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T01:22:07.001571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738812,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66704,"kb_used_data":3648,"kb_used_omap":2700,"kb_used_meta":60339,"kb_avail":94305136,"statfs":{"total":96636764160,"available":96568459264,"internally_reserved":0,"allocated":3735552,"data_stored":2003613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2765235,"internal_metadata":61787725},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738813,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":2175,"kb_used_meta":60864,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":2227362,"internal_metadata":62325598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738810,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":3083,"kb_used_meta":59956,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":3157450,"internal_metadata":61395510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:44:05.645 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2026-02-19T20:44:05.646 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2026-02-19T20:44:05.824 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:44:05.824 INFO:teuthology.orchestra.run.trial073.stderr:dumped all 2026-02-19T20:44:05.833 INFO:teuthology.orchestra.run.trial073.stdout:{"pg_ready":true,"pg_map":{"version":1553,"stamp":"2026-02-19T20:44:05.157968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":200232,"kb_used_data":11064,"kb_used_omap":7959,"kb_used_meta":181160,"kb_avail":282915288,"statfs":{"total":289910292480,"available":289705254912,"internally_reserved":0,"allocated":11329536,"data_stored":7191613,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":8150047,"internal_metadata":185508833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000974"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":914,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.277460+0000","last_change":"2026-02-19T20:07:27.870930+0000","last_active":"2026-02-19T20:43:39.277460+0000","last_peered":"2026-02-19T20:43:39.277460+0000","last_clean":"2026-02-19T20:43:39.277460+0000","last_became_active":"2026-02-19T20:07:25.866966+0000","last_became_peered":"2026-02-19T20:07:25.866966+0000","last_unstale":"2026-02-19T20:43:39.277460+0000","last_undegraded":"2026-02-19T20:43:39.277460+0000","last_fullsized":"2026-02-19T20:43:39.277460+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:08:33.157778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053370000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":914,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.278148+0000","last_change":"2026-02-19T20:07:27.871242+0000","last_active":"2026-02-19T20:43:39.278148+0000","last_peered":"2026-02-19T20:43:39.278148+0000","last_clean":"2026-02-19T20:43:39.278148+0000","last_became_active":"2026-02-19T20:07:25.867278+0000","last_became_peered":"2026-02-19T20:07:25.867278+0000","last_unstale":"2026-02-19T20:43:39.278148+0000","last_undegraded":"2026-02-19T20:43:39.278148+0000","last_fullsized":"2026-02-19T20:43:39.278148+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T05:18:57.062974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081141899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.278308+0000","last_change":"2026-02-19T20:07:27.871411+0000","last_active":"2026-02-19T20:43:39.278308+0000","last_peered":"2026-02-19T20:43:39.278308+0000","last_clean":"2026-02-19T20:43:39.278308+0000","last_became_active":"2026-02-19T20:07:25.867250+0000","last_became_peered":"2026-02-19T20:07:25.867250+0000","last_unstale":"2026-02-19T20:43:39.278308+0000","last_undegraded":"2026-02-19T20:43:39.278308+0000","last_fullsized":"2026-02-19T20:43:39.278308+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:55:29.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094229399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.275854+0000","last_change":"2026-02-19T20:07:27.871199+0000","last_active":"2026-02-19T20:43:39.275854+0000","last_peered":"2026-02-19T20:43:39.275854+0000","last_clean":"2026-02-19T20:43:39.275854+0000","last_became_active":"2026-02-19T20:07:25.867041+0000","last_became_peered":"2026-02-19T20:07:25.867041+0000","last_unstale":"2026-02-19T20:43:39.275854+0000","last_undegraded":"2026-02-19T20:43:39.275854+0000","last_fullsized":"2026-02-19T20:43:39.275854+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:09:11.855713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079016000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:38.593644+0000","last_change":"2026-02-19T20:07:27.869613+0000","last_active":"2026-02-19T20:43:38.593644+0000","last_peered":"2026-02-19T20:43:38.593644+0000","last_clean":"2026-02-19T20:43:38.593644+0000","last_became_active":"2026-02-19T20:07:25.866844+0000","last_became_peered":"2026-02-19T20:07:25.866844+0000","last_unstale":"2026-02-19T20:43:38.593644+0000","last_undegraded":"2026-02-19T20:43:38.593644+0000","last_fullsized":"2026-02-19T20:43:38.593644+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:23.866049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055244400000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.825177+0000","last_change":"2026-02-19T20:07:27.986078+0000","last_active":"2026-02-19T20:43:39.825177+0000","last_peered":"2026-02-19T20:43:39.825177+0000","last_clean":"2026-02-19T20:43:39.825177+0000","last_became_active":"2026-02-19T20:07:25.867756+0000","last_became_peered":"2026-02-19T20:07:25.867756+0000","last_unstale":"2026-02-19T20:43:39.825177+0000","last_undegraded":"2026-02-19T20:43:39.825177+0000","last_fullsized":"2026-02-19T20:43:39.825177+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:22:24.470302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000248385,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.823145+0000","last_change":"2026-02-19T20:07:27.986095+0000","last_active":"2026-02-19T20:43:39.823145+0000","last_peered":"2026-02-19T20:43:39.823145+0000","last_clean":"2026-02-19T20:43:39.823145+0000","last_became_active":"2026-02-19T20:07:25.867520+0000","last_became_peered":"2026-02-19T20:07:25.867520+0000","last_unstale":"2026-02-19T20:43:39.823145+0000","last_undegraded":"2026-02-19T20:43:39.823145+0000","last_fullsized":"2026-02-19T20:43:39.823145+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:16:42.961584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026667900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.277825+0000","last_change":"2026-02-19T20:07:27.870999+0000","last_active":"2026-02-19T20:43:39.277825+0000","last_peered":"2026-02-19T20:43:39.277825+0000","last_clean":"2026-02-19T20:43:39.277825+0000","last_became_active":"2026-02-19T20:07:25.866443+0000","last_became_peered":"2026-02-19T20:07:25.866443+0000","last_unstale":"2026-02-19T20:43:39.277825+0000","last_undegraded":"2026-02-19T20:43:39.277825+0000","last_fullsized":"2026-02-19T20:43:39.277825+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:57:49.025933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059055499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'72","reported_seq":1007,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.274899+0000","last_change":"2026-02-19T20:07:23.081147+0000","last_active":"2026-02-19T20:43:39.274899+0000","last_peered":"2026-02-19T20:43:39.274899+0000","last_clean":"2026-02-19T20:43:39.274899+0000","last_became_active":"2026-02-19T20:07:23.079045+0000","last_became_peered":"2026-02-19T20:07:23.079045+0000","last_unstale":"2026-02-19T20:43:39.274899+0000","last_undegraded":"2026-02-19T20:43:39.274899+0000","last_fullsized":"2026-02-19T20:43:39.274899+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_clean_scrub_stamp":"2026-02-19T20:07:21.856480+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T01:22:07.001571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738812,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66704,"kb_used_data":3648,"kb_used_omap":2700,"kb_used_meta":60339,"kb_avail":94305136,"statfs":{"total":96636764160,"available":96568459264,"internally_reserved":0,"allocated":3735552,"data_stored":2003613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2765235,"internal_metadata":61787725},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738813,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":2175,"kb_used_meta":60864,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":2227362,"internal_metadata":62325598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738810,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":3083,"kb_used_meta":59956,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":3157450,"internal_metadata":61395510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:44:05.835 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2026-02-19T20:44:06.013 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:44:06.014 INFO:teuthology.orchestra.run.trial073.stdout:{"epoch":454,"fsid":"a74de42f-1722-4818-ae18-8cf060b80e9c","created":"2026-02-19T20:07:16.412396+0000","modified":"2026-02-19T20:43:38.588960+0000","last_up_change":"2026-02-19T20:07:20.419166+0000","last_in_change":"2026-02-19T20:07:17.086411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-19T20:07:20.864703+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-19T20:07:24.487762+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0d1fcb52-009c-4059-9bbe-e781f759ca72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6808","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6810","nonce":358751968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6812","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6814","nonce":358751968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6819","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6820","nonce":358751968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6815","nonce":358751968},{"type":"v1","addr":"10.20.193.73:6817","nonce":358751968}]},"public_addr":"10.20.193.73:6810/358751968","cluster_addr":"10.20.193.73:6814/358751968","heartbeat_back_addr":"10.20.193.73:6820/358751968","heartbeat_front_addr":"10.20.193.73:6817/358751968","state":["exists","up"]},{"osd":1,"uuid":"c749e329-91bc-42fa-b289-58af874876ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6809","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6811","nonce":3087991977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6813","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6816","nonce":3087991977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6822","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6823","nonce":3087991977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6818","nonce":3087991977},{"type":"v1","addr":"10.20.193.73:6821","nonce":3087991977}]},"public_addr":"10.20.193.73:6811/3087991977","cluster_addr":"10.20.193.73:6816/3087991977","heartbeat_back_addr":"10.20.193.73:6823/3087991977","heartbeat_front_addr":"10.20.193.73:6821/3087991977","state":["exists","up"]},{"osd":2,"uuid":"77997b69-c81d-4af7-adc0-c3d775ce9285","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6800","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6801","nonce":211675687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6802","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6803","nonce":211675687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6806","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6807","nonce":211675687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.73:6804","nonce":211675687},{"type":"v1","addr":"10.20.193.73:6805","nonce":211675687}]},"public_addr":"10.20.193.73:6801/211675687","cluster_addr":"10.20.193.73:6803/211675687","heartbeat_back_addr":"10.20.193.73:6807/211675687","heartbeat_front_addr":"10.20.193.73:6805/211675687","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:19.035906+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:19.055463+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-19T20:07:18.967627+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-19T20:44:07.022 INFO:tasks.ceph:Scrubbing cluster1.osd.0 2026-02-19T20:44:07.023 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-19T20:44:07.106 INFO:teuthology.orchestra.run.trial073.stdout:{ 2026-02-19T20:44:07.106 INFO:teuthology.orchestra.run.trial073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-19T20:44:07.106 INFO:teuthology.orchestra.run.trial073.stdout:} 2026-02-19T20:44:07.112 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 0 2026-02-19T20:44:07.291 INFO:teuthology.orchestra.run.trial073.stderr:instructed osd(s) 0 to deep-scrub 2026-02-19T20:44:07.300 INFO:tasks.ceph:Scrubbing cluster1.osd.1 2026-02-19T20:44:07.301 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-19T20:44:07.381 INFO:teuthology.orchestra.run.trial073.stdout:{ 2026-02-19T20:44:07.381 INFO:teuthology.orchestra.run.trial073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T20:44:07.381 INFO:teuthology.orchestra.run.trial073.stdout:} 2026-02-19T20:44:07.387 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 1 2026-02-19T20:44:07.560 INFO:teuthology.orchestra.run.trial073.stderr:instructed osd(s) 1 to deep-scrub 2026-02-19T20:44:07.570 INFO:tasks.ceph:Scrubbing cluster1.osd.2 2026-02-19T20:44:07.570 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-19T20:44:07.649 INFO:teuthology.orchestra.run.trial073.stdout:{ 2026-02-19T20:44:07.650 INFO:teuthology.orchestra.run.trial073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-19T20:44:07.650 INFO:teuthology.orchestra.run.trial073.stdout:} 2026-02-19T20:44:07.655 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd deep-scrub 2 2026-02-19T20:44:07.830 INFO:teuthology.orchestra.run.trial073.stderr:instructed osd(s) 2 to deep-scrub 2026-02-19T20:44:07.840 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2026-02-19T20:44:08.017 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:44:08.018 INFO:teuthology.orchestra.run.trial073.stderr:dumped all 2026-02-19T20:44:08.027 INFO:teuthology.orchestra.run.trial073.stdout:{"pg_ready":true,"pg_map":{"version":1554,"stamp":"2026-02-19T20:44:07.158205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":200232,"kb_used_data":11064,"kb_used_omap":7959,"kb_used_meta":181160,"kb_avail":282915288,"statfs":{"total":289910292480,"available":289705254912,"internally_reserved":0,"allocated":11329536,"data_stored":7191613,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":8150047,"internal_metadata":185508833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001057"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":914,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.277460+0000","last_change":"2026-02-19T20:07:27.870930+0000","last_active":"2026-02-19T20:43:39.277460+0000","last_peered":"2026-02-19T20:43:39.277460+0000","last_clean":"2026-02-19T20:43:39.277460+0000","last_became_active":"2026-02-19T20:07:25.866966+0000","last_became_peered":"2026-02-19T20:07:25.866966+0000","last_unstale":"2026-02-19T20:43:39.277460+0000","last_undegraded":"2026-02-19T20:43:39.277460+0000","last_fullsized":"2026-02-19T20:43:39.277460+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T23:08:33.157778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053370000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":914,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.278148+0000","last_change":"2026-02-19T20:07:27.871242+0000","last_active":"2026-02-19T20:43:39.278148+0000","last_peered":"2026-02-19T20:43:39.278148+0000","last_clean":"2026-02-19T20:43:39.278148+0000","last_became_active":"2026-02-19T20:07:25.867278+0000","last_became_peered":"2026-02-19T20:07:25.867278+0000","last_unstale":"2026-02-19T20:43:39.278148+0000","last_undegraded":"2026-02-19T20:43:39.278148+0000","last_fullsized":"2026-02-19T20:43:39.278148+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T05:18:57.062974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081141899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.278308+0000","last_change":"2026-02-19T20:07:27.871411+0000","last_active":"2026-02-19T20:43:39.278308+0000","last_peered":"2026-02-19T20:43:39.278308+0000","last_clean":"2026-02-19T20:43:39.278308+0000","last_became_active":"2026-02-19T20:07:25.867250+0000","last_became_peered":"2026-02-19T20:07:25.867250+0000","last_unstale":"2026-02-19T20:43:39.278308+0000","last_undegraded":"2026-02-19T20:43:39.278308+0000","last_fullsized":"2026-02-19T20:43:39.278308+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:55:29.558488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094229399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.275854+0000","last_change":"2026-02-19T20:07:27.871199+0000","last_active":"2026-02-19T20:43:39.275854+0000","last_peered":"2026-02-19T20:43:39.275854+0000","last_clean":"2026-02-19T20:43:39.275854+0000","last_became_active":"2026-02-19T20:07:25.867041+0000","last_became_peered":"2026-02-19T20:07:25.867041+0000","last_unstale":"2026-02-19T20:43:39.275854+0000","last_undegraded":"2026-02-19T20:43:39.275854+0000","last_fullsized":"2026-02-19T20:43:39.275854+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T20:09:11.855713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079016000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:38.593644+0000","last_change":"2026-02-19T20:07:27.869613+0000","last_active":"2026-02-19T20:43:38.593644+0000","last_peered":"2026-02-19T20:43:38.593644+0000","last_clean":"2026-02-19T20:43:38.593644+0000","last_became_active":"2026-02-19T20:07:25.866844+0000","last_became_peered":"2026-02-19T20:07:25.866844+0000","last_unstale":"2026-02-19T20:43:38.593644+0000","last_undegraded":"2026-02-19T20:43:38.593644+0000","last_fullsized":"2026-02-19T20:43:38.593644+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:19:23.866049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055244400000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":915,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.825177+0000","last_change":"2026-02-19T20:07:27.986078+0000","last_active":"2026-02-19T20:43:39.825177+0000","last_peered":"2026-02-19T20:43:39.825177+0000","last_clean":"2026-02-19T20:43:39.825177+0000","last_became_active":"2026-02-19T20:07:25.867756+0000","last_became_peered":"2026-02-19T20:07:25.867756+0000","last_unstale":"2026-02-19T20:43:39.825177+0000","last_undegraded":"2026-02-19T20:43:39.825177+0000","last_fullsized":"2026-02-19T20:43:39.825177+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:22:24.470302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000248385,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.823145+0000","last_change":"2026-02-19T20:07:27.986095+0000","last_active":"2026-02-19T20:43:39.823145+0000","last_peered":"2026-02-19T20:43:39.823145+0000","last_clean":"2026-02-19T20:43:39.823145+0000","last_became_active":"2026-02-19T20:07:25.867520+0000","last_became_peered":"2026-02-19T20:07:25.867520+0000","last_unstale":"2026-02-19T20:43:39.823145+0000","last_undegraded":"2026-02-19T20:43:39.823145+0000","last_fullsized":"2026-02-19T20:43:39.823145+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:16:42.961584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026667900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":916,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.277825+0000","last_change":"2026-02-19T20:07:27.870999+0000","last_active":"2026-02-19T20:43:39.277825+0000","last_peered":"2026-02-19T20:43:39.277825+0000","last_clean":"2026-02-19T20:43:39.277825+0000","last_became_active":"2026-02-19T20:07:25.866443+0000","last_became_peered":"2026-02-19T20:07:25.866443+0000","last_unstale":"2026-02-19T20:43:39.277825+0000","last_undegraded":"2026-02-19T20:43:39.277825+0000","last_fullsized":"2026-02-19T20:43:39.277825+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:24.859954+0000","last_clean_scrub_stamp":"2026-02-19T20:07:24.859954+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:57:49.025933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059055499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'72","reported_seq":1007,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:43:39.274899+0000","last_change":"2026-02-19T20:07:23.081147+0000","last_active":"2026-02-19T20:43:39.274899+0000","last_peered":"2026-02-19T20:43:39.274899+0000","last_clean":"2026-02-19T20:43:39.274899+0000","last_became_active":"2026-02-19T20:07:23.079045+0000","last_became_peered":"2026-02-19T20:07:23.079045+0000","last_unstale":"2026-02-19T20:43:39.274899+0000","last_undegraded":"2026-02-19T20:43:39.274899+0000","last_fullsized":"2026-02-19T20:43:39.274899+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:07:21.856480+0000","last_clean_scrub_stamp":"2026-02-19T20:07:21.856480+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T01:22:07.001571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66704,"kb_used_data":3648,"kb_used_omap":2700,"kb_used_meta":60339,"kb_avail":94305136,"statfs":{"total":96636764160,"available":96568459264,"internally_reserved":0,"allocated":3735552,"data_stored":2003613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2765235,"internal_metadata":61787725},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738814,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":2175,"kb_used_meta":60864,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":2227362,"internal_metadata":62325598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738811,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":3083,"kb_used_meta":59956,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":3157450,"internal_metadata":61395510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:44:08.028 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-19T20:07:24.859954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=24, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=44, tm_sec=6, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:44:08.028 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-19T20:07:24.859954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=24, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=44, tm_sec=6, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:44:08.028 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-19T20:07:24.859954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=24, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=44, tm_sec=6, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:44:08.028 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-19T20:07:24.859954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=24, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=44, tm_sec=6, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:44:08.028 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-19T20:07:24.859954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=24, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=44, tm_sec=6, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:44:08.028 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-19T20:07:24.859954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=24, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=44, tm_sec=6, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:44:08.028 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-19T20:07:24.859954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=24, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=44, tm_sec=6, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:44:08.028 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-19T20:07:24.859954+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=24, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=44, tm_sec=6, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:44:08.028 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-19T20:07:21.856480+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=7, tm_sec=21, tm_wday=3, tm_yday=50, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=19, tm_hour=20, tm_min=44, tm_sec=6, tm_wday=3, tm_yday=50, tm_isdst=0) 2026-02-19T20:44:08.028 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-19T20:44:28.029 DEBUG:teuthology.orchestra.run.trial073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2026-02-19T20:44:28.206 INFO:teuthology.orchestra.run.trial073.stdout: 2026-02-19T20:44:28.207 INFO:teuthology.orchestra.run.trial073.stderr:dumped all 2026-02-19T20:44:28.215 INFO:teuthology.orchestra.run.trial073.stdout:{"pg_ready":true,"pg_map":{"version":1564,"stamp":"2026-02-19T20:44:27.159972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":200232,"kb_used_data":11064,"kb_used_omap":7959,"kb_used_meta":181160,"kb_avail":282915288,"statfs":{"total":289910292480,"available":289705254912,"internally_reserved":0,"allocated":11329536,"data_stored":7191613,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":8150047,"internal_metadata":185508833},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000943"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":923,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:44:12.055145+0000","last_change":"2026-02-19T20:44:12.055089+0000","last_active":"2026-02-19T20:44:12.055145+0000","last_peered":"2026-02-19T20:44:12.055145+0000","last_clean":"2026-02-19T20:44:12.055145+0000","last_became_active":"2026-02-19T20:07:25.866966+0000","last_became_peered":"2026-02-19T20:07:25.866966+0000","last_unstale":"2026-02-19T20:44:12.055145+0000","last_undegraded":"2026-02-19T20:44:12.055145+0000","last_fullsized":"2026-02-19T20:44:12.055145+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:44:12.055011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:44:12.055011+0000","last_clean_scrub_stamp":"2026-02-19T20:44:12.055011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:59:27.438960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053370000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":923,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:44:13.070461+0000","last_change":"2026-02-19T20:44:13.070426+0000","last_active":"2026-02-19T20:44:13.070461+0000","last_peered":"2026-02-19T20:44:13.070461+0000","last_clean":"2026-02-19T20:44:13.070461+0000","last_became_active":"2026-02-19T20:07:25.867278+0000","last_became_peered":"2026-02-19T20:07:25.867278+0000","last_unstale":"2026-02-19T20:44:13.070461+0000","last_undegraded":"2026-02-19T20:44:13.070461+0000","last_fullsized":"2026-02-19T20:44:13.070461+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:44:13.070372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:44:13.070372+0000","last_clean_scrub_stamp":"2026-02-19T20:44:13.070372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:54:12.921665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081141899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":923,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:44:11.052893+0000","last_change":"2026-02-19T20:44:11.052893+0000","last_active":"2026-02-19T20:44:11.052893+0000","last_peered":"2026-02-19T20:44:11.052893+0000","last_clean":"2026-02-19T20:44:11.052893+0000","last_became_active":"2026-02-19T20:07:25.867250+0000","last_became_peered":"2026-02-19T20:07:25.867250+0000","last_unstale":"2026-02-19T20:44:11.052893+0000","last_undegraded":"2026-02-19T20:44:11.052893+0000","last_fullsized":"2026-02-19T20:44:11.052893+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:44:11.052850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:44:11.052850+0000","last_clean_scrub_stamp":"2026-02-19T20:44:11.052850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T06:18:39.309883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094229399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":923,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:44:10.097242+0000","last_change":"2026-02-19T20:44:10.097242+0000","last_active":"2026-02-19T20:44:10.097242+0000","last_peered":"2026-02-19T20:44:10.097242+0000","last_clean":"2026-02-19T20:44:10.097242+0000","last_became_active":"2026-02-19T20:07:25.867041+0000","last_became_peered":"2026-02-19T20:07:25.867041+0000","last_unstale":"2026-02-19T20:44:10.097242+0000","last_undegraded":"2026-02-19T20:44:10.097242+0000","last_fullsized":"2026-02-19T20:44:10.097242+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:44:10.097184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:44:10.097184+0000","last_clean_scrub_stamp":"2026-02-19T20:44:10.097184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T01:42:16.352339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079016000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"454'3","reported_seq":925,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:44:07.802294+0000","last_change":"2026-02-19T20:44:07.802294+0000","last_active":"2026-02-19T20:44:07.802294+0000","last_peered":"2026-02-19T20:44:07.802294+0000","last_clean":"2026-02-19T20:44:07.802294+0000","last_became_active":"2026-02-19T20:07:25.866844+0000","last_became_peered":"2026-02-19T20:07:25.866844+0000","last_unstale":"2026-02-19T20:44:07.802294+0000","last_undegraded":"2026-02-19T20:44:07.802294+0000","last_fullsized":"2026-02-19T20:44:07.802294+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"454'3","last_scrub_stamp":"2026-02-19T20:44:07.802254+0000","last_deep_scrub":"454'3","last_deep_scrub_stamp":"2026-02-19T20:44:07.802254+0000","last_clean_scrub_stamp":"2026-02-19T20:44:07.802254+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T03:27:59.929288+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00055244400000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":923,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:44:08.798778+0000","last_change":"2026-02-19T20:44:08.798778+0000","last_active":"2026-02-19T20:44:08.798778+0000","last_peered":"2026-02-19T20:44:08.798778+0000","last_clean":"2026-02-19T20:44:08.798778+0000","last_became_active":"2026-02-19T20:07:25.867756+0000","last_became_peered":"2026-02-19T20:07:25.867756+0000","last_unstale":"2026-02-19T20:44:08.798778+0000","last_undegraded":"2026-02-19T20:44:08.798778+0000","last_fullsized":"2026-02-19T20:44:08.798778+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:44:08.798051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:44:08.798051+0000","last_clean_scrub_stamp":"2026-02-19T20:44:08.798051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T02:03:12.780925+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000248385,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":924,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:44:09.791333+0000","last_change":"2026-02-19T20:44:09.791333+0000","last_active":"2026-02-19T20:44:09.791333+0000","last_peered":"2026-02-19T20:44:09.791333+0000","last_clean":"2026-02-19T20:44:09.791333+0000","last_became_active":"2026-02-19T20:07:25.867520+0000","last_became_peered":"2026-02-19T20:07:25.867520+0000","last_unstale":"2026-02-19T20:44:09.791333+0000","last_undegraded":"2026-02-19T20:44:09.791333+0000","last_fullsized":"2026-02-19T20:44:09.791333+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-19T20:44:09.791298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-19T20:44:09.791298+0000","last_clean_scrub_stamp":"2026-02-19T20:44:09.791298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T04:59:33.233893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00026667900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"454'2","reported_seq":925,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:44:09.118706+0000","last_change":"2026-02-19T20:44:09.118706+0000","last_active":"2026-02-19T20:44:09.118706+0000","last_peered":"2026-02-19T20:44:09.118706+0000","last_clean":"2026-02-19T20:44:09.118706+0000","last_became_active":"2026-02-19T20:07:25.866443+0000","last_became_peered":"2026-02-19T20:07:25.866443+0000","last_unstale":"2026-02-19T20:44:09.118706+0000","last_undegraded":"2026-02-19T20:44:09.118706+0000","last_fullsized":"2026-02-19T20:44:09.118706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"454'2","last_scrub_stamp":"2026-02-19T20:44:09.118672+0000","last_deep_scrub":"454'2","last_deep_scrub_stamp":"2026-02-19T20:44:09.118672+0000","last_clean_scrub_stamp":"2026-02-19T20:44:09.118672+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-20T22:15:15.365408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059055499999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"454'74","reported_seq":1017,"reported_epoch":454,"state":"active+clean","last_fresh":"2026-02-19T20:44:08.101614+0000","last_change":"2026-02-19T20:44:08.101614+0000","last_active":"2026-02-19T20:44:08.101614+0000","last_peered":"2026-02-19T20:44:08.101614+0000","last_clean":"2026-02-19T20:44:08.101614+0000","last_became_active":"2026-02-19T20:07:23.079045+0000","last_became_peered":"2026-02-19T20:07:23.079045+0000","last_unstale":"2026-02-19T20:44:08.101614+0000","last_undegraded":"2026-02-19T20:44:08.101614+0000","last_fullsized":"2026-02-19T20:44:08.101614+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"454'74","last_scrub_stamp":"2026-02-19T20:44:08.101564+0000","last_deep_scrub":"454'74","last_deep_scrub_stamp":"2026-02-19T20:44:08.101564+0000","last_clean_scrub_stamp":"2026-02-19T20:44:08.101564+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-21T01:00:43.754857+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12962,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738817,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66704,"kb_used_data":3648,"kb_used_omap":2700,"kb_used_meta":60339,"kb_avail":94305136,"statfs":{"total":96636764160,"available":96568459264,"internally_reserved":0,"allocated":3735552,"data_stored":2003613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":2765235,"internal_metadata":61787725},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738818,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":2175,"kb_used_meta":60864,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":2227362,"internal_metadata":62325598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738815,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66764,"kb_used_data":3708,"kb_used_omap":3083,"kb_used_meta":59956,"kb_avail":94305076,"statfs":{"total":96636764160,"available":96568397824,"internally_reserved":0,"allocated":3796992,"data_stored":2594000,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":3157450,"internal_metadata":61395510},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6481,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-19T20:44:28.218 DEBUG:teuthology.orchestra.run.trial073:> sudo ceph --cluster cluster1 config set global mon_health_to_clog false 2026-02-19T20:44:28.405 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-19T20:44:28.406 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-19T20:44:28.406 DEBUG:tasks.ceph.cluster1.osd.0:waiting for process to exit 2026-02-19T20:44:28.406 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:28.451 INFO:tasks.ceph.cluster1.osd.0:Stopped 2026-02-19T20:44:28.451 DEBUG:tasks.ceph.cluster1.osd.1:waiting for process to exit 2026-02-19T20:44:28.452 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:28.497 INFO:tasks.ceph.cluster1.osd.1:Stopped 2026-02-19T20:44:28.497 DEBUG:tasks.ceph.cluster1.osd.2:waiting for process to exit 2026-02-19T20:44:28.497 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:28.547 INFO:tasks.ceph.cluster1.osd.2:Stopped 2026-02-19T20:44:28.547 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-19T20:44:28.548 DEBUG:tasks.ceph.cluster1.mgr.x:waiting for process to exit 2026-02-19T20:44:28.548 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:28.580 INFO:tasks.ceph.cluster1.mgr.x:Stopped 2026-02-19T20:44:28.580 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-19T20:44:28.580 DEBUG:tasks.ceph.cluster1.mon.a:waiting for process to exit 2026-02-19T20:44:28.580 INFO:teuthology.orchestra.run:waiting for 300 2026-02-19T20:44:28.631 INFO:tasks.ceph.cluster1.mon.a:Stopped 2026-02-19T20:44:28.631 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-19T20:44:28.631 DEBUG:teuthology.orchestra.run.trial073:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cluster1.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(OSD_SLOW_PING_TIME' | head -n 1 2026-02-19T20:44:28.697 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-0 on ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:44:28.698 DEBUG:teuthology.orchestra.run.trial073:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-0 2026-02-19T20:44:28.771 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-1 on ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:44:28.771 DEBUG:teuthology.orchestra.run.trial073:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-1 2026-02-19T20:44:28.849 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/cluster1-2 on ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:44:28.850 DEBUG:teuthology.orchestra.run.trial073:> sync && sudo umount -f /var/lib/ceph/osd/cluster1-2 2026-02-19T20:44:28.928 INFO:tasks.ceph:Archiving mon data... 2026-02-19T20:44:28.928 DEBUG:teuthology.misc:Transferring archived files from trial073:/var/lib/ceph/mon/cluster1-a to /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606/data/cluster1.mon.a.tgz 2026-02-19T20:44:28.928 DEBUG:teuthology.orchestra.run.trial073:> mktemp 2026-02-19T20:44:28.983 INFO:teuthology.orchestra.run.trial073.stdout:/tmp/tmp.2XvKgYxt11 2026-02-19T20:44:28.983 DEBUG:teuthology.orchestra.run.trial073:> sudo tar cz -f - -C /var/lib/ceph/mon/cluster1-a -- . > /tmp/tmp.2XvKgYxt11 2026-02-19T20:44:29.102 DEBUG:teuthology.orchestra.run.trial073:> sudo chmod 0666 /tmp/tmp.2XvKgYxt11 2026-02-19T20:44:29.183 DEBUG:teuthology.orchestra.remote:trial073:/tmp/tmp.2XvKgYxt11 is 405KB 2026-02-19T20:44:29.271 DEBUG:teuthology.orchestra.run.trial073:> rm -fr /tmp/tmp.2XvKgYxt11 2026-02-19T20:44:29.288 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-19T20:44:29.288 DEBUG:teuthology.orchestra.run.trial073:> sudo rm -rf -- /etc/ceph/cluster1.conf /etc/ceph/cluster1.keyring /home/ubuntu/cephtest/cluster1.data /home/ubuntu/cephtest/cluster1.monmap /home/ubuntu/cephtest/../*.pid 2026-02-19T20:44:29.331 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -rf -- /etc/ceph/cluster1.conf /etc/ceph/cluster1.keyring /home/ubuntu/cephtest/cluster1.data /home/ubuntu/cephtest/cluster1.monmap /home/ubuntu/cephtest/../*.pid 2026-02-19T20:44:29.410 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-19T20:44:29.426 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-19T20:44:29.426 INFO:tasks.ceph:Archiving crash dumps... 2026-02-19T20:44:29.430 DEBUG:teuthology.misc:Transferring archived files from trial073:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606/remote/trial073/crash 2026-02-19T20:44:29.432 DEBUG:teuthology.orchestra.run.trial073:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-19T20:44:29.474 DEBUG:teuthology.misc:Transferring archived files from trial080:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606/remote/trial080/crash 2026-02-19T20:44:29.475 DEBUG:teuthology.orchestra.run.trial080:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-19T20:44:29.500 INFO:tasks.ceph:Compressing logs... 2026-02-19T20:44:29.500 DEBUG:teuthology.orchestra.run.trial073:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-19T20:44:29.520 DEBUG:teuthology.orchestra.run.trial080:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-19T20:44:29.540 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43114.log 2026-02-19T20:44:29.556 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46357.log 2026-02-19T20:44:29.556 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.43114.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43114.log.gz 2026-02-19T20:44:29.556 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46356.log 2026-02-19T20:44:29.556 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46357.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48636.log 2026-02-19T20:44:29.556 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46357.log.gz 2026-02-19T20:44:29.556 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42893.log 2026-02-19T20:44:29.556 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46356.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46356.log.gz 2026-02-19T20:44:29.556 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48636.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48636.log.gz 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46010.log 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.42893.log: gzip -5 --verbose -- /var/log/ceph/cluster2-mgr.x.log 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42893.log.gz 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-mon.a.log 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46010.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46010.log.gz 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster2-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46108.log 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cluster1-osd.2.log 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42661.log 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46108.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46108.log.gz 2026-02-19T20:44:29.557 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-osd.2.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42987.log 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.43248.log 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.42661.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42661.log.gz 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.42987.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42987.log.gz 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-osd.0.log 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45896.log 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.43248.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.43248.log.gz 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-osd.0.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45801.log 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.45896.log: gzip -5 --verbose -- /var/log/ceph/cluster1-osd.1.log 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr: 80.6% -- replaced with /var/log/ceph/cluster1-client.admin.45896.log.gz 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.45801.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45801.log.gz 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48538.log 2026-02-19T20:44:29.558 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-osd.1.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48733.log 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48538.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46209.log 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48538.log.gz 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48733.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46451.log 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48733.log.gz 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46209.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42756.log 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46209.log.gz 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46451.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46451.log.gz 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46210.log 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.42756.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48244.log 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42756.log.gz 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46210.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46500.log 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46210.log.gz 2026-02-19T20:44:29.559 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48244.log: 0.0%gzip -- replaced with /var/log/ceph/cluster1-client.admin.48244.log.gz 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.46358.log 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.admin.46500.log: -- /var/log/ceph/cluster1-client.admin.48342.log 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46500.log.gz 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46358.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46358.log.gz 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-mgr.x.log 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48342.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48342.log.gz 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48686.log 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48489.log 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48686.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48146.log 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48686.log.gz 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48489.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46059.log 2026-02-19T20:44:29.560 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48489.log.gz 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48146.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48195.log 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48146.log.gz 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46059.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46059.log.gz 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45961.log 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48195.log: gzip -5 --verbose -- /var/log/ceph/cluster1.audit.log 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48195.log.gz 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.45961.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.45961.log.gz 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48391.log 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48293.log 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1.audit.log: /var/log/ceph/cluster1-client.admin.48391.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.46211.log 2026-02-19T20:44:29.561 INFO:teuthology.orchestra.run.trial073.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48391.log.gz 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48293.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48293.log.gz 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.42940.log 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46211.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46211.log.gz 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48440.log 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1.log 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.42940.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.42940.log.gz 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48440.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.37803.log 0.0% 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr: -- replaced with /var/log/ceph/cluster1-client.admin.48440.log.gz 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.45871.log 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr: 93.0%gzip -- replaced with /var/log/ceph/cluster1.audit.log.gz -5 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr: --verbose -- /var/log/ceph/cluster1-client.admin.46549.log 2026-02-19T20:44:29.562 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/ceph.tmp-client.admin.37803.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.37803.log.gz 2026-02-19T20:44:29.563 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.45871.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/cluster1-client.admin.45871.log.gz -- 2026-02-19T20:44:29.563 INFO:teuthology.orchestra.run.trial073.stderr: /var/log/ceph/cluster1-client.admin.48587.log 2026-02-19T20:44:29.563 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.46549.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.46549.log.gz 2026-02-19T20:44:29.563 INFO:teuthology.orchestra.run.trial073.stderr:/var/log/ceph/cluster1-client.admin.48587.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48587.log.gz 2026-02-19T20:44:29.563 INFO:teuthology.orchestra.run.trial073.stderr: 88.2% -- replaced with /var/log/ceph/cluster1.log.gz 2026-02-19T20:44:29.573 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61652.log 2026-02-19T20:44:29.573 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70384.log 2026-02-19T20:44:29.573 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61652.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66127.log 2026-02-19T20:44:29.573 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61652.log.gz 2026-02-19T20:44:29.573 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70384.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83861.log 2026-02-19T20:44:29.573 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66127.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73753.log 2026-02-19T20:44:29.573 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.66127.log.gz 2026-02-19T20:44:29.574 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83861.log: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.83861.log.gz 2026-02-19T20:44:29.574 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54121.log 2026-02-19T20:44:29.574 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73753.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68210.log 2026-02-19T20:44:29.574 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.70384.log.gz 2026-02-19T20:44:29.574 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54121.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67198.log 2026-02-19T20:44:29.574 INFO:teuthology.orchestra.run.trial080.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.73753.log.gz 2026-02-19T20:44:29.574 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68210.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63368.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.68210.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67198.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67437.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.3.47995.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63368.log: /var/log/ceph/cluster2-client.mirror.67437.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70672.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.63368.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.3.47995.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62134.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.67437.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70672.log: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.67198.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46956.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62134.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60218.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46956.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52987.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.62134.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46956.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.70672.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60218.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47490.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52987.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55742.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60218.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 92.9% -- replaced with /var/log/ceph/cluster1-client.mirror.3.47995.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57955.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.47490.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47490.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55742.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54403.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57955.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75186.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.57955.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54403.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66320.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.43964.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75186.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.54403.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66320.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57679.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.43964.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.66320.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55851.log 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 29.2% -- replaced with /var/log/ceph/cluster2-client.admin.43964.log.gz 2026-02-19T20:44:29.587 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.75186.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57679.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62935.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57679.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75207.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55851.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.52987.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62935.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63870.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster2-client.mirror.62935.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55742.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75207.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59830.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63870.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.75207.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74236.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.63870.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56426.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59830.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.59830.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74236.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48158.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74236.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56426.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84391.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48158.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.56426.log.gzgzip 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78532.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84391.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64185.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.84391.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78532.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58135.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78532.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64185.log: gzip 93.4% -5 -- replaced with /var/log/ceph/cluster2-client.mirror.55851.log.gz --verbose 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster2-client.mirror.60452.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:gzip 87.4% -5 --verbose -- replaced with /var/log/ceph/cluster2-client.mirror.64185.log.gz -- 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster2-client.mirror.2.48123.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58135.log: /var/log/ceph/cluster2-client.mirror.60452.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52365.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60452.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.2.48123.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68961.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 95.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48158.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52365.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76550.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68961.log: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.52365.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47343.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68961.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76550.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64233.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.47343.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70139.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47343.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64233.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63848.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.64233.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76550.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70139.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77857.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63848.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62869.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77857.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54034.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77857.log.gz 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58193.log 2026-02-19T20:44:29.588 INFO:teuthology.orchestra.run.trial080.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.70139.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62869.log: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.63848.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54034.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69206.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.62869.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58193.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67678.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58193.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69206.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75765.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67678.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.69206.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71294.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.67678.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75765.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84744.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75765.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71294.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76911.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84744.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.71294.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58076.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84744.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76911.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48423.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58076.log: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58076.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62782.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.54034.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.48423.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62827.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48423.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76911.log.gz 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66462.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62782.log: /var/log/ceph/cluster2-client.mirror.62827.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80919.log 2026-02-19T20:44:29.589 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62782.log.gz 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66462.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.62827.log.gz 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64555.log 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.66462.log.gz 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80919.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55020.log 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.80919.log.gz 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64555.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76787.log 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.64555.log.gz 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74938.log 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55020.log: /var/log/ceph/cluster2-client.mirror.76787.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.56000.log 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.55020.log.gz 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74938.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55333.log 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.56000.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52636.log 2026-02-19T20:44:29.590 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76787.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.74938.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55333.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67157.log 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52636.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68740.log 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67157.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73779.log 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.67157.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68740.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78375.log 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.68740.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73779.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56166.log 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.73779.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78375.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71989.log 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78375.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56166.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85329.log 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56166.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71989.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79888.log 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.56000.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.55333.log.gz -- replaced with /var/log/ceph/cluster1-client.mirror.71989.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85329.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75517.log 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85329.log.gz 2026-02-19T20:44:29.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79888.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76447.log 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75517.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.79888.log.gz 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.52727.log 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.52636.log.gz 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.75517.log.gz 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76447.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84411.log 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76447.log.gz 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.52727.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61999.log 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84411.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.admin.52727.log.gz 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74628.log 2026-02-19T20:44:29.592 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61999.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61734.log 2026-02-19T20:44:29.593 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.61999.log.gz 2026-02-19T20:44:29.593 INFO:teuthology.orchestra.run.trial080.stderr: 90.7% -- replaced with /var/log/ceph/cluster2-client.mirror.84411.log.gz 2026-02-19T20:44:29.593 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74628.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58939.log 2026-02-19T20:44:29.593 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61734.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80726.log 2026-02-19T20:44:29.593 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.61734.log.gz 2026-02-19T20:44:29.593 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58939.log: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.74628.log.gz 2026-02-19T20:44:29.593 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82973.log 2026-02-19T20:44:29.593 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80726.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80368.log 2026-02-19T20:44:29.594 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58939.log.gz 2026-02-19T20:44:29.594 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82973.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76116.log 2026-02-19T20:44:29.594 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.82973.log.gz 2026-02-19T20:44:29.594 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80368.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54150.log 2026-02-19T20:44:29.594 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.80368.log.gz 2026-02-19T20:44:29.594 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76116.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47076.log 2026-02-19T20:44:29.595 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.54150.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58294.log 2026-02-19T20:44:29.595 INFO:teuthology.orchestra.run.trial080.stderr: 87.8% -- replaced with /var/log/ceph/cluster1-client.admin.54150.log.gz 2026-02-19T20:44:29.595 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47076.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47076.log.gz 2026-02-19T20:44:29.595 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50555.log 2026-02-19T20:44:29.595 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.76116.log.gz 2026-02-19T20:44:29.595 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58294.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66610.log 2026-02-19T20:44:29.595 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.50555.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63506.log 2026-02-19T20:44:29.596 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66610.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83394.log 2026-02-19T20:44:29.596 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.66610.log.gz 2026-02-19T20:44:29.596 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63506.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64056.log 2026-02-19T20:44:29.596 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.63506.log.gz 2026-02-19T20:44:29.596 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83394.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73717.log 2026-02-19T20:44:29.596 INFO:teuthology.orchestra.run.trial080.stderr: 84.8% -- replaced with /var/log/ceph/cluster2-client.mirror.83394.log.gz 2026-02-19T20:44:29.596 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64056.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83964.log 2026-02-19T20:44:29.597 INFO:teuthology.orchestra.run.trial080.stderr: 87.7% -- replaced with /var/log/ceph/cluster1-client.mirror.64056.log.gz 2026-02-19T20:44:29.597 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73717.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85127.log 2026-02-19T20:44:29.597 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.73717.log.gz 2026-02-19T20:44:29.597 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.83964.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.60528.log 2026-02-19T20:44:29.597 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.83964.log.gz/var/log/ceph/cluster2-client.admin.85127.log: 2026-02-19T20:44:29.597 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85127.log.gz 2026-02-19T20:44:29.597 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54283.log 2026-02-19T20:44:29.597 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.50555.log.gz 2026-02-19T20:44:29.597 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.2.60528.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48532.log 2026-02-19T20:44:29.598 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54283.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56594.log 2026-02-19T20:44:29.598 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48532.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.54283.log.gz 2026-02-19T20:44:29.598 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49027.log 2026-02-19T20:44:29.598 INFO:teuthology.orchestra.run.trial080.stderr: 3.5% -- replaced with /var/log/ceph/cluster1-client.mirror.48532.log.gz 2026-02-19T20:44:29.598 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56594.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72862.log 2026-02-19T20:44:29.598 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.56594.log.gz 2026-02-19T20:44:29.599 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49027.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46494.log 2026-02-19T20:44:29.599 INFO:teuthology.orchestra.run.trial080.stderr: 2.3% -- replaced with /var/log/ceph/cluster2-client.mirror.49027.log.gz 2026-02-19T20:44:29.599 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72862.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80409.log 2026-02-19T20:44:29.599 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46494.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75414.log 2026-02-19T20:44:29.599 INFO:teuthology.orchestra.run.trial080.stderr: 80.5% -- replaced with /var/log/ceph/cluster2-client.admin.46494.log.gz 2026-02-19T20:44:29.599 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80704.log 2026-02-19T20:44:29.599 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80409.log: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.72862.log.gz 2026-02-19T20:44:29.600 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75414.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61925.log 2026-02-19T20:44:29.600 INFO:teuthology.orchestra.run.trial080.stderr: 86.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80409.log.gz 2026-02-19T20:44:29.600 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.75414.log.gz 2026-02-19T20:44:29.600 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80704.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84600.log 2026-02-19T20:44:29.600 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.80704.log.gz 2026-02-19T20:44:29.600 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61925.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73606.log 2026-02-19T20:44:29.600 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.84600.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.61925.log.gz 2026-02-19T20:44:29.600 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2.log 2026-02-19T20:44:29.600 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.84600.log.gz 2026-02-19T20:44:29.601 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73606.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77385.log 2026-02-19T20:44:29.601 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.73606.log.gz 2026-02-19T20:44:29.601 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67722.log 2026-02-19T20:44:29.601 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77385.log: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.77385.log.gz 2026-02-19T20:44:29.602 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69524.log 2026-02-19T20:44:29.602 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67722.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62909.log 2026-02-19T20:44:29.603 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69524.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50486.log 2026-02-19T20:44:29.603 INFO:teuthology.orchestra.run.trial080.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.67722.log.gz 2026-02-19T20:44:29.603 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62909.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71029.log 2026-02-19T20:44:29.603 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50486.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.62909.log.gz 2026-02-19T20:44:29.603 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52405.log 2026-02-19T20:44:29.603 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.50486.log.gz 2026-02-19T20:44:29.603 INFO:teuthology.orchestra.run.trial080.stderr: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.69524.log.gz 2026-02-19T20:44:29.604 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71029.log: 88.1% -- replaced with /var/log/ceph/cluster2.log.gz 2026-02-19T20:44:29.604 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.48651.log 2026-02-19T20:44:29.604 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.71029.log.gz 2026-02-19T20:44:29.604 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52405.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56942.log 2026-02-19T20:44:29.604 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.48651.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.48651.log.gz 2026-02-19T20:44:29.604 INFO:teuthology.orchestra.run.trial080.stderr: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.52405.log.gz 2026-02-19T20:44:29.604 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49112.log 2026-02-19T20:44:29.604 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56942.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74380.log 2026-02-19T20:44:29.604 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49112.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56942.log.gz 2026-02-19T20:44:29.605 INFO:teuthology.orchestra.run.trial080.stderr: 64.3% -- replaced with /var/log/ceph/cluster2-client.mirror.49112.log.gz 2026-02-19T20:44:29.605 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67636.log 2026-02-19T20:44:29.605 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74380.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77565.log 2026-02-19T20:44:29.605 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67636.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68439.log 2026-02-19T20:44:29.605 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77565.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75558.log 2026-02-19T20:44:29.605 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74380.log.gz 2026-02-19T20:44:29.605 INFO:teuthology.orchestra.run.trial080.stderr: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.77565.log.gz 2026-02-19T20:44:29.606 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68439.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69503.log 2026-02-19T20:44:29.606 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.68439.log.gz 2026-02-19T20:44:29.606 INFO:teuthology.orchestra.run.trial080.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.67636.log.gz 2026-02-19T20:44:29.606 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75558.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55182.log 2026-02-19T20:44:29.607 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69503.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51343.log 2026-02-19T20:44:29.607 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55182.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.69503.log.gz 2026-02-19T20:44:29.607 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64776.log 2026-02-19T20:44:29.607 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55182.log.gz 2026-02-19T20:44:29.607 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51343.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.75558.log.gz 2026-02-19T20:44:29.607 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47294.log 2026-02-19T20:44:29.607 INFO:teuthology.orchestra.run.trial080.stderr: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.51343.log.gz 2026-02-19T20:44:29.607 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64776.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69660.log 2026-02-19T20:44:29.608 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.64776.log.gz 2026-02-19T20:44:29.608 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47294.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47294.log.gz 2026-02-19T20:44:29.608 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71567.log 2026-02-19T20:44:29.608 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69660.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78262.log 2026-02-19T20:44:29.608 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.69660.log.gz 2026-02-19T20:44:29.608 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71567.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55894.log 2026-02-19T20:44:29.608 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.71567.log.gz 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78262.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.78262.log.gz 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.53754.log 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose --/var/log/ceph/cluster2-client.mirror.55894.log: /var/log/ceph/cluster2-client.mirror.56838.log 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.53754.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57286.log 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr: 87.7% -- replaced with /var/log/ceph/cluster1-client.admin.53754.log.gz 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56838.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74132.log 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.56838.log.gz 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57286.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75868.log 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57286.log.gz 2026-02-19T20:44:29.609 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74132.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80555.log 2026-02-19T20:44:29.610 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82535.log 2026-02-19T20:44:29.610 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75868.log: /var/log/ceph/cluster2-client.mirror.80555.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74132.log.gz 2026-02-19T20:44:29.610 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82993.log 2026-02-19T20:44:29.610 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82535.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56616.log 2026-02-19T20:44:29.610 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.75868.log.gz 2026-02-19T20:44:29.611 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.82535.log.gz 2026-02-19T20:44:29.611 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82993.log: 90.8% -- replaced with /var/log/ceph/cluster2-client.mirror.80555.log.gz 2026-02-19T20:44:29.611 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52528.log 2026-02-19T20:44:29.611 INFO:teuthology.orchestra.run.trial080.stderr: 86.7% -- replaced with /var/log/ceph/cluster2-client.mirror.82993.log.gz 2026-02-19T20:44:29.611 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55894.log.gz 2026-02-19T20:44:29.611 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56616.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54509.log 2026-02-19T20:44:29.611 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52528.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82890.log 2026-02-19T20:44:29.611 INFO:teuthology.orchestra.run.trial080.stderr: 88.5% -- replaced with /var/log/ceph/cluster1-client.mirror.52528.log.gz 2026-02-19T20:44:29.612 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54509.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71873.log 2026-02-19T20:44:29.612 INFO:teuthology.orchestra.run.trial080.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54509.log.gz 2026-02-19T20:44:29.612 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.82890.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70505.log 2026-02-19T20:44:29.612 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.82890.log.gz 2026-02-19T20:44:29.612 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71873.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74814.log 2026-02-19T20:44:29.612 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70505.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71736.log 2026-02-19T20:44:29.612 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74814.log: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.70505.log.gz 2026-02-19T20:44:29.612 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65083.log 2026-02-19T20:44:29.613 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.71873.log.gz 2026-02-19T20:44:29.613 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71736.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52211.log 2026-02-19T20:44:29.613 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.71736.log.gz 2026-02-19T20:44:29.613 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65083.log: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.74814.log.gz 2026-02-19T20:44:29.613 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77340.log 2026-02-19T20:44:29.613 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52211.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59591.log 2026-02-19T20:44:29.613 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77340.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.52211.log.gz 2026-02-19T20:44:29.614 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77340.log.gz 2026-02-19T20:44:29.614 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.65083.log.gzgzip 2026-02-19T20:44:29.614 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77452.log 2026-02-19T20:44:29.614 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59591.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56206.log 2026-02-19T20:44:29.614 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59591.log.gz 2026-02-19T20:44:29.614 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77452.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83840.log 2026-02-19T20:44:29.614 INFO:teuthology.orchestra.run.trial080.stderr: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.77452.log.gz 2026-02-19T20:44:29.614 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56206.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61424.log 2026-02-19T20:44:29.615 INFO:teuthology.orchestra.run.trial080.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56206.log.gz 2026-02-19T20:44:29.615 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83840.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61180.log 2026-02-19T20:44:29.615 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61424.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83549.log 2026-02-19T20:44:29.615 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78802.log 2026-02-19T20:44:29.615 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61180.log: 90.5% -- replaced with /var/log/ceph/cluster2-client.mirror.83840.log.gz 2026-02-19T20:44:29.615 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.83549.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.61180.log.gzgzip 2026-02-19T20:44:29.615 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48583.log 2026-02-19T20:44:29.615 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.83549.log.gz 2026-02-19T20:44:29.616 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78802.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83507.log 2026-02-19T20:44:29.616 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78802.log.gz 2026-02-19T20:44:29.616 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48583.log: 63.6% -- replaced with /var/log/ceph/cluster1-client.mirror.48583.log.gz 2026-02-19T20:44:29.616 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48617.log 2026-02-19T20:44:29.616 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65653.log 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.83507.log: /var/log/ceph/cluster1-client.mirror.48617.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81839.log 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.83507.log.gz 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr: 54.5% -- replaced with /var/log/ceph/cluster1-client.mirror.48617.log.gz 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65653.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58690.log 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65653.log.gz 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.81839.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58880.log 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.81839.log.gz 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58690.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59120.log 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58880.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56306.log 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59120.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47075.log 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56306.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.59120.log.gz 2026-02-19T20:44:29.617 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83985.log 2026-02-19T20:44:29.618 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56306.log.gz 2026-02-19T20:44:29.618 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47075.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47075.log.gz 2026-02-19T20:44:29.618 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66975.log 2026-02-19T20:44:29.618 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83985.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63530.log 2026-02-19T20:44:29.618 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66975.log: 93.5% -- replaced with /var/log/ceph/cluster2-client.mirror.2.48123.log.gz 2026-02-19T20:44:29.618 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.83985.log.gz 2026-02-19T20:44:29.618 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64368.log 2026-02-19T20:44:29.618 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.66975.log.gz 2026-02-19T20:44:29.619 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63530.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73851.log 2026-02-19T20:44:29.619 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.63530.log.gz 2026-02-19T20:44:29.619 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64368.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78960.log 2026-02-19T20:44:29.619 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73851.log: gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.64368.log.gz -- 2026-02-19T20:44:29.619 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster2-client.mirror.83250.log 2026-02-19T20:44:29.619 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.73851.log.gz 2026-02-19T20:44:29.619 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78960.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65629.log 2026-02-19T20:44:29.619 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78960.log.gz 2026-02-19T20:44:29.620 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83250.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79703.log 2026-02-19T20:44:29.620 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.83250.log.gz 2026-02-19T20:44:29.620 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65629.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73899.log 2026-02-19T20:44:29.620 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.65629.log.gz 2026-02-19T20:44:29.620 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79703.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80007.log 2026-02-19T20:44:29.620 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.79703.log.gz 2026-02-19T20:44:29.621 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73899.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57308.log 2026-02-19T20:44:29.621 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.73899.log.gz 2026-02-19T20:44:29.621 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80007.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71832.log 2026-02-19T20:44:29.621 INFO:teuthology.orchestra.run.trial080.stderr: 87.7% -- replaced with /var/log/ceph/cluster2-client.mirror.80007.log.gz 2026-02-19T20:44:29.621 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57308.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53640.log 2026-02-19T20:44:29.621 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71832.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69455.log 2026-02-19T20:44:29.621 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.71832.log.gz 2026-02-19T20:44:29.621 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53640.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70295.log 2026-02-19T20:44:29.622 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74608.log 2026-02-19T20:44:29.622 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69455.log: /var/log/ceph/cluster1-client.mirror.70295.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55267.log 2026-02-19T20:44:29.622 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.69455.log.gz 2026-02-19T20:44:29.622 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74608.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.70295.log.gz 2026-02-19T20:44:29.622 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73654.log 2026-02-19T20:44:29.623 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.74608.log.gz 2026-02-19T20:44:29.623 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55267.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67796.log 2026-02-19T20:44:29.623 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73654.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66294.log 2026-02-19T20:44:29.623 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67796.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.73654.log.gz 2026-02-19T20:44:29.623 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55158.log 2026-02-19T20:44:29.623 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.67796.log.gz 2026-02-19T20:44:29.624 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79949.log 2026-02-19T20:44:29.624 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66294.log: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.53640.log.gz 2026-02-19T20:44:29.624 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55158.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81184.log 2026-02-19T20:44:29.624 INFO:teuthology.orchestra.run.trial080.stderr: 89.1% -- replaced with /var/log/ceph/cluster1-client.mirror.55158.log.gz 2026-02-19T20:44:29.624 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66294.log.gz 2026-02-19T20:44:29.624 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79949.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63783.log 2026-02-19T20:44:29.624 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81184.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64576.log 2026-02-19T20:44:29.625 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63783.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70800.log 2026-02-19T20:44:29.625 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.63783.log.gz 2026-02-19T20:44:29.625 INFO:teuthology.orchestra.run.trial080.stderr: 90.7% -- replaced with /var/log/ceph/cluster2-client.mirror.79949.log.gz 2026-02-19T20:44:29.625 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46954.log 2026-02-19T20:44:29.625 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64576.log: /var/log/ceph/cluster1-client.mirror.70800.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58723.log 2026-02-19T20:44:29.625 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.70800.log.gz 2026-02-19T20:44:29.625 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46954.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65287.log 2026-02-19T20:44:29.626 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46954.log.gz 2026-02-19T20:44:29.626 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55267.log.gz 2026-02-19T20:44:29.626 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57716.log 2026-02-19T20:44:29.626 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58723.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58723.log.gz 2026-02-19T20:44:29.626 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65287.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.64576.log.gz 2026-02-19T20:44:29.626 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61493.log 2026-02-19T20:44:29.626 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.65287.log.gz 2026-02-19T20:44:29.626 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57716.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85623.log 2026-02-19T20:44:29.627 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61493.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62431.log 2026-02-19T20:44:29.627 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57716.log.gz 2026-02-19T20:44:29.627 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61493.log.gz 2026-02-19T20:44:29.627 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85623.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85623.log.gz 2026-02-19T20:44:29.627 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64346.log 2026-02-19T20:44:29.627 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85525.log 2026-02-19T20:44:29.627 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62431.log: /var/log/ceph/cluster1-client.mirror.64346.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51506.log 2026-02-19T20:44:29.628 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.62431.log.gz 2026-02-19T20:44:29.628 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85525.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85525.log.gz 2026-02-19T20:44:29.628 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46399.log 2026-02-19T20:44:29.628 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.51506.log: 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.64346.log.gz 2026-02-19T20:44:29.628 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49205.log 2026-02-19T20:44:29.628 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.51506.log.gz 2026-02-19T20:44:29.628 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46399.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46399.log.gz 2026-02-19T20:44:29.628 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56326.log 2026-02-19T20:44:29.629 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose --/var/log/ceph/cluster2-client.mirror.49205.log: /var/log/ceph/cluster1-client.mirror.70094.log 2026-02-19T20:44:29.629 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56326.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.49205.log.gz 2026-02-19T20:44:29.629 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53921.log 2026-02-19T20:44:29.629 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56326.log.gz 2026-02-19T20:44:29.629 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.70094.log: -- /var/log/ceph/cluster2-client.mirror.64863.log 2026-02-19T20:44:29.629 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53921.log: gzip 87.6% -5 -- replaced with /var/log/ceph/cluster1-client.mirror.70094.log.gz --verbose 2026-02-19T20:44:29.630 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster1-client.mirror.1.47901.log 2026-02-19T20:44:29.630 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster1-client.mirror.53921.log.gz 2026-02-19T20:44:29.630 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64863.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65215.log 2026-02-19T20:44:29.630 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65770.log 2026-02-19T20:44:29.630 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.64863.log.gz 2026-02-19T20:44:29.630 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.1.47901.log: /var/log/ceph/cluster1-client.mirror.65215.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68788.log 2026-02-19T20:44:29.631 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.65215.log.gz 2026-02-19T20:44:29.631 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65770.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62734.log 2026-02-19T20:44:29.631 INFO:teuthology.orchestra.run.trial080.stderr:gzip/var/log/ceph/cluster1-client.mirror.68788.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56962.log 2026-02-19T20:44:29.631 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62734.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72429.log 2026-02-19T20:44:29.631 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68788.log.gz 2026-02-19T20:44:29.631 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.65770.log.gz 2026-02-19T20:44:29.632 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.62734.log.gz 2026-02-19T20:44:29.632 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56962.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.43591.log 2026-02-19T20:44:29.632 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.56962.log.gz 2026-02-19T20:44:29.632 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54355.log 2026-02-19T20:44:29.632 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72429.log: gzip -5/var/log/ceph/cluster2-client.admin.43591.log: --verbose -- /var/log/ceph/cluster2-client.mirror.53207.log 2026-02-19T20:44:29.632 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.43591.log.gz 2026-02-19T20:44:29.632 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54355.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55419.log 2026-02-19T20:44:29.632 INFO:teuthology.orchestra.run.trial080.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.72429.log.gz 2026-02-19T20:44:29.633 INFO:teuthology.orchestra.run.trial080.stderr: 89.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54355.log.gz 2026-02-19T20:44:29.633 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53207.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85770.log 2026-02-19T20:44:29.633 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55419.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79617.log 2026-02-19T20:44:29.633 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85770.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.53274.log 2026-02-19T20:44:29.633 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85770.log.gz 2026-02-19T20:44:29.634 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79617.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78420.log 2026-02-19T20:44:29.634 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.79617.log.gz 2026-02-19T20:44:29.634 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.53274.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68463.log 2026-02-19T20:44:29.634 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster1-client.mirror.1.47901.log.gz 2026-02-19T20:44:29.634 INFO:teuthology.orchestra.run.trial080.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.admin.53274.log.gz 2026-02-19T20:44:29.634 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78420.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74649.log 2026-02-19T20:44:29.634 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78420.log.gz 2026-02-19T20:44:29.634 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68463.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71188.log 2026-02-19T20:44:29.635 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.68463.log.gz 2026-02-19T20:44:29.635 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74649.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65749.log 2026-02-19T20:44:29.635 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74649.log.gz 2026-02-19T20:44:29.635 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71188.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74277.log 2026-02-19T20:44:29.635 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65749.log: 88.4%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.71188.log.gz 2026-02-19T20:44:29.635 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.2.84219.log 2026-02-19T20:44:29.636 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65749.log.gz 2026-02-19T20:44:29.636 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74277.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75827.log 2026-02-19T20:44:29.636 INFO:teuthology.orchestra.run.trial080.stderr: 93.0% -- replaced with /var/log/ceph/cluster1-client.mirror.2.60528.log.gz 2026-02-19T20:44:29.636 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55419.log.gz 2026-02-19T20:44:29.636 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.74277.log.gz 2026-02-19T20:44:29.636 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.2.84219.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68327.log 2026-02-19T20:44:29.636 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75827.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75290.log 2026-02-19T20:44:29.636 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.75827.log.gz 2026-02-19T20:44:29.637 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68327.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79531.log 2026-02-19T20:44:29.637 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84308.log 2026-02-19T20:44:29.637 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75290.log: /var/log/ceph/cluster2-client.mirror.79531.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56186.log 2026-02-19T20:44:29.637 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.75290.log.gz 2026-02-19T20:44:29.637 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% -- replaced with /var/log/ceph/cluster2-client.mirror.2.84219.log.gz 2026-02-19T20:44:29.637 INFO:teuthology.orchestra.run.trial080.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.79531.log.gz 2026-02-19T20:44:29.638 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84308.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58411.log 2026-02-19T20:44:29.638 INFO:teuthology.orchestra.run.trial080.stderr: 92.1% -- replaced with /var/log/ceph/cluster2-client.mirror.68327.log.gz 2026-02-19T20:44:29.638 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.84308.log.gz 2026-02-19T20:44:29.638 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56186.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66041.log 2026-02-19T20:44:29.638 INFO:teuthology.orchestra.run.trial080.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56186.log.gz 2026-02-19T20:44:29.638 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58411.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73128.log 2026-02-19T20:44:29.638 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66041.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69932.log 2026-02-19T20:44:29.639 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58411.log.gz 2026-02-19T20:44:29.639 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73128.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.66041.log.gz 2026-02-19T20:44:29.639 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63119.log 2026-02-19T20:44:29.639 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.73128.log.gz 2026-02-19T20:44:29.639 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69932.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83572.log 2026-02-19T20:44:29.639 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.69932.log.gz 2026-02-19T20:44:29.639 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63119.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56366.log 2026-02-19T20:44:29.640 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61684.log 2026-02-19T20:44:29.640 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.63119.log.gz 2026-02-19T20:44:29.640 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.83572.log: 86.2% -- replaced with /var/log/ceph/cluster1-client.mirror.83572.log.gz 2026-02-19T20:44:29.640 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56366.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78015.log 2026-02-19T20:44:29.640 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56366.log.gz 2026-02-19T20:44:29.640 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61684.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61293.log 2026-02-19T20:44:29.640 INFO:teuthology.orchestra.run.trial080.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.61684.log.gz 2026-02-19T20:44:29.641 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78015.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71712.log 2026-02-19T20:44:29.641 INFO:teuthology.orchestra.run.trial080.stderr: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.78015.log.gz 2026-02-19T20:44:29.641 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61293.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69610.log 2026-02-19T20:44:29.641 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61293.log.gz 2026-02-19T20:44:29.641 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71712.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53620.log 2026-02-19T20:44:29.641 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.71712.log.gz 2026-02-19T20:44:29.641 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76953.log 2026-02-19T20:44:29.642 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69610.log: /var/log/ceph/cluster2-client.mirror.53620.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.57589.log 2026-02-19T20:44:29.642 INFO:teuthology.orchestra.run.trial080.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69610.log.gz 2026-02-19T20:44:29.642 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76953.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82432.log 2026-02-19T20:44:29.642 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.76953.log.gz 2026-02-19T20:44:29.642 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.0.57589.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47120.log 2026-02-19T20:44:29.643 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.82432.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58272.log 2026-02-19T20:44:29.643 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.82432.log.gz 2026-02-19T20:44:29.643 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47120.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58959.log 2026-02-19T20:44:29.643 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47120.log.gz 2026-02-19T20:44:29.643 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58272.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55636.log 2026-02-19T20:44:29.643 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58272.log.gz 2026-02-19T20:44:29.644 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58959.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66912.log 2026-02-19T20:44:29.644 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58959.log.gz 2026-02-19T20:44:29.644 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55636.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81270.log 2026-02-19T20:44:29.644 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.53620.log.gz 2026-02-19T20:44:29.644 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66912.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62616.log 2026-02-19T20:44:29.644 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.66912.log.gz 2026-02-19T20:44:29.645 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.81270.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60837.log 2026-02-19T20:44:29.645 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.81270.log.gz 2026-02-19T20:44:29.645 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62616.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59083.log 2026-02-19T20:44:29.645 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60837.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.62616.log.gz 2026-02-19T20:44:29.645 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.2.72961.log 2026-02-19T20:44:29.645 INFO:teuthology.orchestra.run.trial080.stderr: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.60837.log.gz 2026-02-19T20:44:29.645 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59083.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67415.log 2026-02-19T20:44:29.646 INFO:teuthology.orchestra.run.trial080.stderr: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59083.log.gz 2026-02-19T20:44:29.646 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.2.72961.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76364.log 2026-02-19T20:44:29.646 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67415.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83820.log 2026-02-19T20:44:29.646 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76364.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55636.log.gz 2026-02-19T20:44:29.646 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52658.log 2026-02-19T20:44:29.647 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69254.log 2026-02-19T20:44:29.647 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83820.log: /var/log/ceph/cluster2-client.mirror.52658.log: gzip -5 87.6% --verbose -- replaced with /var/log/ceph/cluster2-client.mirror.83820.log.gz -- 2026-02-19T20:44:29.647 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster1-client.mirror.52146.log 2026-02-19T20:44:29.647 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.67415.log.gz 2026-02-19T20:44:29.647 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76364.log.gz 2026-02-19T20:44:29.647 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69254.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70824.log 2026-02-19T20:44:29.647 INFO:teuthology.orchestra.run.trial080.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.52658.log.gz 2026-02-19T20:44:29.648 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.69254.log.gz 2026-02-19T20:44:29.648 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52146.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79050.log 2026-02-19T20:44:29.648 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70824.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60415.log 2026-02-19T20:44:29.648 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79050.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.70824.log.gz 2026-02-19T20:44:29.648 INFO:teuthology.orchestra.run.trial080.stderr: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.79050.log.gz 2026-02-19T20:44:29.648 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82453.log 2026-02-19T20:44:29.648 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67772.log 2026-02-19T20:44:29.649 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60415.log: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.60415.log.gz 2026-02-19T20:44:29.649 INFO:teuthology.orchestra.run.trial080.stderr: 93.2% -- replaced with /var/log/ceph/cluster2-client.mirror.0.57589.log.gz 2026-02-19T20:44:29.649 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82453.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72575.log 2026-02-19T20:44:29.649 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.82453.log.gz 2026-02-19T20:44:29.649 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67772.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.48019.log 2026-02-19T20:44:29.649 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.67772.log.gz 2026-02-19T20:44:29.649 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72575.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69120.log 2026-02-19T20:44:29.650 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.0.48019.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82388.log 2026-02-19T20:44:29.650 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.72575.log.gz 2026-02-19T20:44:29.650 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69120.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75724.log 2026-02-19T20:44:29.650 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.69120.log.gz 2026-02-19T20:44:29.650 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82388.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.76615.log 2026-02-19T20:44:29.650 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75724.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.82388.log.gz 2026-02-19T20:44:29.650 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48019.log 2026-02-19T20:44:29.651 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.52146.log.gz 2026-02-19T20:44:29.651 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75724.log.gz 2026-02-19T20:44:29.651 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.0.76615.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71220.log 2026-02-19T20:44:29.651 INFO:teuthology.orchestra.run.trial080.stderr: 91.1% -- replaced with /var/log/ceph/cluster2-client.mirror.0.48019.log.gz 2026-02-19T20:44:29.651 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48019.log: 77.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48019.log.gz 2026-02-19T20:44:29.651 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80326.log 2026-02-19T20:44:29.651 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67969.log 2026-02-19T20:44:29.652 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71220.log: /var/log/ceph/cluster2-client.mirror.80326.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60432.log 2026-02-19T20:44:29.652 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.71220.log.gz 2026-02-19T20:44:29.652 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67969.log: 86.7% -- replaced with /var/log/ceph/cluster2-client.mirror.80326.log.gz 2026-02-19T20:44:29.652 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60732.log 2026-02-19T20:44:29.652 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.67969.log.gz 2026-02-19T20:44:29.652 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60432.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.3.51850.log 2026-02-19T20:44:29.653 INFO:teuthology.orchestra.run.trial080.stderr: 88.6%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.60432.log.gz -5 2026-02-19T20:44:29.653 INFO:teuthology.orchestra.run.trial080.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.78915.log 2026-02-19T20:44:29.653 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60732.log: /var/log/ceph/cluster1-client.mirror.3.51850.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67395.log 2026-02-19T20:44:29.653 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60732.log.gz 2026-02-19T20:44:29.653 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78915.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76158.log 2026-02-19T20:44:29.653 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78915.log.gz 2026-02-19T20:44:29.653 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67395.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77880.log 2026-02-19T20:44:29.654 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76158.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68258.log 2026-02-19T20:44:29.654 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.0.76615.log.gz 2026-02-19T20:44:29.654 INFO:teuthology.orchestra.run.trial080.stderr: 87.7% -- replaced with /var/log/ceph/cluster2-client.mirror.76158.log.gz 2026-02-19T20:44:29.654 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47784.log 2026-02-19T20:44:29.654 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77880.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.2.72961.log.gz 2026-02-19T20:44:29.655 INFO:teuthology.orchestra.run.trial080.stderr: 86.5%/var/log/ceph/cluster1-client.mirror.68258.log: -- replaced with /var/log/ceph/cluster1-client.mirror.77880.log.gz 2026-02-19T20:44:29.655 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59747.log 2026-02-19T20:44:29.655 INFO:teuthology.orchestra.run.trial080.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.67395.log.gz 2026-02-19T20:44:29.655 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.68258.log.gz 2026-02-19T20:44:29.655 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47784.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58351.log 2026-02-19T20:44:29.655 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47784.log.gz 2026-02-19T20:44:29.655 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59747.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82188.log 2026-02-19T20:44:29.655 INFO:teuthology.orchestra.run.trial080.stderr: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.59747.log.gz 2026-02-19T20:44:29.656 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58351.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50864.log 2026-02-19T20:44:29.656 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58351.log.gz 2026-02-19T20:44:29.656 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82188.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66840.log 2026-02-19T20:44:29.656 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.82188.log.gz 2026-02-19T20:44:29.656 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.50864.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77610.log 2026-02-19T20:44:29.656 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66840.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74546.log 2026-02-19T20:44:29.657 INFO:teuthology.orchestra.run.trial080.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.50864.log.gz 2026-02-19T20:44:29.657 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.66840.log.gz 2026-02-19T20:44:29.657 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77610.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64596.log 2026-02-19T20:44:29.657 INFO:teuthology.orchestra.run.trial080.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.77610.log.gz 2026-02-19T20:44:29.657 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74546.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69756.log 2026-02-19T20:44:29.657 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.74546.log.gz 2026-02-19T20:44:29.658 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64596.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81924.log 2026-02-19T20:44:29.658 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69756.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54257.log 2026-02-19T20:44:29.658 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81924.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75496.log 2026-02-19T20:44:29.658 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54257.log: 86.9% -- replaced with /var/log/ceph/cluster2-client.mirror.81924.log.gz 2026-02-19T20:44:29.658 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.64596.log.gz 2026-02-19T20:44:29.659 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85079.log 2026-02-19T20:44:29.659 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75496.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79455.log 2026-02-19T20:44:29.659 INFO:teuthology.orchestra.run.trial080.stderr: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.54257.log.gz 2026-02-19T20:44:29.659 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85079.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73279.log 2026-02-19T20:44:29.659 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85079.log.gz 2026-02-19T20:44:29.660 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79455.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.43638.log 2026-02-19T20:44:29.660 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.75496.log.gz 2026-02-19T20:44:29.660 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.79455.log.gz 2026-02-19T20:44:29.660 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73279.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56246.log 2026-02-19T20:44:29.660 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.43638.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.43638.log.gz 2026-02-19T20:44:29.660 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84029.log 2026-02-19T20:44:29.660 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73279.log.gz 2026-02-19T20:44:29.660 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56246.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78465.log 2026-02-19T20:44:29.661 INFO:teuthology.orchestra.run.trial080.stderr: 87.9% -- replaced with /var/log/ceph/cluster2-client.mirror.56246.log.gz 2026-02-19T20:44:29.661 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.84029.log: gzip -5 --verbose -- /var/log/ceph/cluster2.audit.log 2026-02-19T20:44:29.661 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.84029.log.gz 2026-02-19T20:44:29.661 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78465.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58113.log 2026-02-19T20:44:29.661 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78465.log.gz 2026-02-19T20:44:29.661 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2.audit.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54729.log 2026-02-19T20:44:29.662 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58113.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55482.log 2026-02-19T20:44:29.662 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58113.log.gz 2026-02-19T20:44:29.662 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54729.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81904.log 2026-02-19T20:44:29.662 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55482.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69278.log 2026-02-19T20:44:29.663 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81904.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57876.log 2026-02-19T20:44:29.663 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81904.log.gz 2026-02-19T20:44:29.663 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69278.log: 93.3% -- replaced with /var/log/ceph/cluster2.audit.log.gz 2026-02-19T20:44:29.663 INFO:teuthology.orchestra.run.trial080.stderr: 87.8% -- replaced with /var/log/ceph/cluster1-client.mirror.69278.log.gz 2026-02-19T20:44:29.664 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57876.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77632.log 2026-02-19T20:44:29.664 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.57876.log.gz 2026-02-19T20:44:29.664 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54379.log 2026-02-19T20:44:29.664 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77632.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68282.log 2026-02-19T20:44:29.664 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.77632.log.gz 2026-02-19T20:44:29.664 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54379.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56467.log 2026-02-19T20:44:29.665 INFO:teuthology.orchestra.run.trial080.stderr: 89.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54379.log.gz 2026-02-19T20:44:29.665 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68282.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78690.log 2026-02-19T20:44:29.665 INFO:teuthology.orchestra.run.trial080.stderr: 87.0% -- replaced with /var/log/ceph/cluster1-client.mirror.68282.log.gz 2026-02-19T20:44:29.665 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56467.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76240.log 2026-02-19T20:44:29.665 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.56467.log.gz 2026-02-19T20:44:29.665 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55482.log.gz 2026-02-19T20:44:29.666 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78690.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75951.log 2026-02-19T20:44:29.666 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78690.log.gz 2026-02-19T20:44:29.666 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76240.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71125.log 2026-02-19T20:44:29.666 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75951.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63032.log 2026-02-19T20:44:29.666 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75951.log.gz 2026-02-19T20:44:29.666 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71125.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62324.log 2026-02-19T20:44:29.667 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.71125.log.gz 2026-02-19T20:44:29.667 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.76240.log.gz 2026-02-19T20:44:29.667 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63032.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85721.log 2026-02-19T20:44:29.667 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.63032.log.gz 2026-02-19T20:44:29.667 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62324.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61114.log 2026-02-19T20:44:29.667 INFO:teuthology.orchestra.run.trial080.stderr: 87.6%/var/log/ceph/cluster2-client.admin.85721.log: -- replaced with /var/log/ceph/cluster1-client.mirror.62324.log.gz 2026-02-19T20:44:29.667 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85721.log.gz 2026-02-19T20:44:29.667 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72310.log 2026-02-19T20:44:29.668 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61114.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83055.log 2026-02-19T20:44:29.668 INFO:teuthology.orchestra.run.trial073.stderr: 91.0% -- replaced with /var/log/ceph/cluster2-mgr.x.log.gz 2026-02-19T20:44:29.668 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72310.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61114.log.gz 2026-02-19T20:44:29.668 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50462.log 2026-02-19T20:44:29.668 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.72310.log.gz 2026-02-19T20:44:29.668 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83055.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78127.log 2026-02-19T20:44:29.668 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.83055.log.gz 2026-02-19T20:44:29.669 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50462.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62387.log 2026-02-19T20:44:29.669 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.50462.log.gz 2026-02-19T20:44:29.669 INFO:teuthology.orchestra.run.trial073.stderr: 91.1% -- replaced with /var/log/ceph/cluster1-mgr.x.log.gz 2026-02-19T20:44:29.669 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78127.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67326.log 2026-02-19T20:44:29.669 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78127.log.gz 2026-02-19T20:44:29.669 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62387.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76302.log 2026-02-19T20:44:29.669 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67326.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.62387.log.gz 2026-02-19T20:44:29.670 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.69756.log 2026-02-19T20:44:29.670 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.67326.log.gz/var/log/ceph/cluster2-client.mirror.76302.log: 2026-02-19T20:44:29.670 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83528.log 2026-02-19T20:44:29.670 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.0.69756.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82617.log 2026-02-19T20:44:29.671 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83528.log: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.69756.log.gz 2026-02-19T20:44:29.671 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72638.log 2026-02-19T20:44:29.671 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76302.log.gz 2026-02-19T20:44:29.671 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82617.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.83528.log.gz 2026-02-19T20:44:29.671 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69972.log 2026-02-19T20:44:29.671 INFO:teuthology.orchestra.run.trial080.stderr: 86.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82617.log.gz 2026-02-19T20:44:29.671 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72638.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63807.log 2026-02-19T20:44:29.671 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.72638.log.gz 2026-02-19T20:44:29.672 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69972.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82110.log 2026-02-19T20:44:29.672 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63807.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58552.log 2026-02-19T20:44:29.672 INFO:teuthology.orchestra.run.trial080.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.69972.log.gz 2026-02-19T20:44:29.672 INFO:teuthology.orchestra.run.trial080.stderr: 87.8% -- replaced with /var/log/ceph/cluster1-client.mirror.63807.log.gz 2026-02-19T20:44:29.672 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82110.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66792.log 2026-02-19T20:44:29.672 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.82110.log.gz 2026-02-19T20:44:29.672 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58552.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64483.log 2026-02-19T20:44:29.672 INFO:teuthology.orchestra.run.trial080.stderr: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58552.log.gz 2026-02-19T20:44:29.673 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66792.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65954.log 2026-02-19T20:44:29.673 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.66792.log.gz 2026-02-19T20:44:29.673 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64483.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.83394.log 2026-02-19T20:44:29.673 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.64483.log.gz 2026-02-19T20:44:29.673 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65954.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60874.log 2026-02-19T20:44:29.674 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.65954.log.gz 2026-02-19T20:44:29.674 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.2.83394.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77139.log 2026-02-19T20:44:29.674 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60874.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48095.log 2026-02-19T20:44:29.674 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60874.log.gz 2026-02-19T20:44:29.674 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.77139.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57997.log 2026-02-19T20:44:29.675 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48095.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.77139.log.gz 2026-02-19T20:44:29.675 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81376.log 2026-02-19T20:44:29.675 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57997.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74835.log 2026-02-19T20:44:29.675 INFO:teuthology.orchestra.run.trial080.stderr: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.57997.log.gz 2026-02-19T20:44:29.675 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81376.log: 94.1% -- replaced with /var/log/ceph/cluster1-client.mirror.48095.log.gz 2026-02-19T20:44:29.675 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79252.log 2026-02-19T20:44:29.675 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81376.log.gz 2026-02-19T20:44:29.676 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74835.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50388.log 2026-02-19T20:44:29.676 INFO:teuthology.orchestra.run.trial080.stderr: 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.2.83394.log.gz 2026-02-19T20:44:29.676 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.74835.log.gz 2026-02-19T20:44:29.676 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79252.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68875.log 2026-02-19T20:44:29.676 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.79252.log.gz 2026-02-19T20:44:29.676 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.50388.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58473.log 2026-02-19T20:44:29.676 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68875.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80791.log 89.0% 2026-02-19T20:44:29.677 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.50388.log.gz 2026-02-19T20:44:29.677 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.68875.log.gz 2026-02-19T20:44:29.677 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58473.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72037.log 2026-02-19T20:44:29.677 INFO:teuthology.orchestra.run.trial080.stderr: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58473.log.gz 2026-02-19T20:44:29.677 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80791.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72085.log 2026-02-19T20:44:29.677 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72037.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70118.log 2026-02-19T20:44:29.677 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.80791.log.gz 2026-02-19T20:44:29.678 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.72037.log.gz 2026-02-19T20:44:29.678 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72085.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53683.log 2026-02-19T20:44:29.678 INFO:teuthology.orchestra.run.trial080.stderr: 87.8% -- replaced with /var/log/ceph/cluster1-client.mirror.72085.log.gz 2026-02-19T20:44:29.678 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70118.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55353.log 2026-02-19T20:44:29.678 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53683.log: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.70118.log.gz 2026-02-19T20:44:29.678 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46559.log 2026-02-19T20:44:29.679 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55353.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73152.log 2026-02-19T20:44:29.679 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46559.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72674.log 2026-02-19T20:44:29.679 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46559.log.gz 2026-02-19T20:44:29.679 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73152.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76137.log 2026-02-19T20:44:29.679 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.73152.log.gz 2026-02-19T20:44:29.679 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72674.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79297.log 2026-02-19T20:44:29.680 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76137.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.72674.log.gz 2026-02-19T20:44:29.680 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48566.log 2026-02-19T20:44:29.680 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79297.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74422.log 2026-02-19T20:44:29.680 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76137.log.gz 2026-02-19T20:44:29.680 INFO:teuthology.orchestra.run.trial080.stderr: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.79297.log.gz 2026-02-19T20:44:29.681 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48566.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74360.log 2026-02-19T20:44:29.681 INFO:teuthology.orchestra.run.trial080.stderr: 63.6% -- replaced with /var/log/ceph/cluster1-client.mirror.48566.log.gz 2026-02-19T20:44:29.681 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74422.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81062.log 2026-02-19T20:44:29.681 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74422.log.gz 2026-02-19T20:44:29.681 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74360.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.43833.log 2026-02-19T20:44:29.681 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74360.log.gz 2026-02-19T20:44:29.681 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81062.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67591.log 2026-02-19T20:44:29.682 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.53683.log.gz 2026-02-19T20:44:29.682 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55353.log.gz 2026-02-19T20:44:29.682 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.43833.log: 86.7% -- replaced with /var/log/ceph/cluster2-client.mirror.81062.log.gz 2026-02-19T20:44:29.682 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78555.log 2026-02-19T20:44:29.682 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.43833.log.gz 2026-02-19T20:44:29.682 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67591.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79682.log 2026-02-19T20:44:29.682 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.67591.log.gz 2026-02-19T20:44:29.682 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78555.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50575.log 2026-02-19T20:44:29.682 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78555.log.gz 2026-02-19T20:44:29.683 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79682.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58056.log 2026-02-19T20:44:29.683 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.79682.log.gz 2026-02-19T20:44:29.683 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50575.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69708.log 2026-02-19T20:44:29.683 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73425.log 2026-02-19T20:44:29.684 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58056.log: /var/log/ceph/cluster1-client.mirror.69708.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61197.log 2026-02-19T20:44:29.684 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.69708.log.gz 2026-02-19T20:44:29.684 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73425.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.52312.log 2026-02-19T20:44:29.684 INFO:teuthology.orchestra.run.trial080.stderr: 87.6%/var/log/ceph/cluster2-client.mirror.61197.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66175.log 2026-02-19T20:44:29.684 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.73425.log.gz 2026-02-19T20:44:29.684 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.52312.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55871.log 2026-02-19T20:44:29.685 INFO:teuthology.orchestra.run.trial080.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.61197.log.gz 2026-02-19T20:44:29.685 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66175.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.admin.52312.log.gz 2026-02-19T20:44:29.685 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53366.log 2026-02-19T20:44:29.685 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.66175.log.gz 2026-02-19T20:44:29.685 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55871.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65603.log 2026-02-19T20:44:29.685 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53366.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53599.log 2026-02-19T20:44:29.686 INFO:teuthology.orchestra.run.trial080.stderr: 88.4%gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.65603.log: -- /var/log/ceph/cluster1-client.mirror.67109.log 2026-02-19T20:44:29.686 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.53366.log.gz 2026-02-19T20:44:29.686 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.50575.log.gz 2026-02-19T20:44:29.686 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53599.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85280.log 2026-02-19T20:44:29.686 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.65603.log.gz 2026-02-19T20:44:29.686 INFO:teuthology.orchestra.run.trial080.stderr: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.53599.log.gz 2026-02-19T20:44:29.686 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67109.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82409.log 2026-02-19T20:44:29.687 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.67109.log.gz 2026-02-19T20:44:29.687 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85280.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56855.log 2026-02-19T20:44:29.687 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85280.log.gz 2026-02-19T20:44:29.687 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.82409.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50438.log 2026-02-19T20:44:29.687 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.82409.log.gz 2026-02-19T20:44:29.687 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56855.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62112.log 2026-02-19T20:44:29.688 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50438.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75744.log 2026-02-19T20:44:29.688 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56855.log.gz 2026-02-19T20:44:29.688 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.50438.log.gz 2026-02-19T20:44:29.688 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62112.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80898.log 2026-02-19T20:44:29.688 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55871.log.gz 2026-02-19T20:44:29.688 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75744.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55957.log 2026-02-19T20:44:29.689 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80898.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75972.log 2026-02-19T20:44:29.689 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.80898.log.gz 2026-02-19T20:44:29.689 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55957.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80857.log 2026-02-19T20:44:29.689 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.62112.log.gz 2026-02-19T20:44:29.689 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.75744.log.gz 2026-02-19T20:44:29.689 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75972.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79970.log 2026-02-19T20:44:29.690 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80857.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84840.log 2026-02-19T20:44:29.690 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75972.log.gz 2026-02-19T20:44:29.690 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.80857.log.gz/var/log/ceph/cluster2-client.mirror.79970.log: 2026-02-19T20:44:29.690 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55505.log 2026-02-19T20:44:29.690 INFO:teuthology.orchestra.run.trial080.stderr: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.79970.log.gz 2026-02-19T20:44:29.690 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84840.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73401.log 2026-02-19T20:44:29.690 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84840.log.gz 2026-02-19T20:44:29.690 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55505.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83308.log 2026-02-19T20:44:29.691 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73401.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.51753.log 2026-02-19T20:44:29.691 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.73401.log.gz 2026-02-19T20:44:29.691 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83308.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47735.log 2026-02-19T20:44:29.691 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.51753.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74980.log 2026-02-19T20:44:29.691 INFO:teuthology.orchestra.run.trial080.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.83308.log.gz 2026-02-19T20:44:29.691 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47735.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.admin.51753.log.gz 2026-02-19T20:44:29.692 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47735.log.gz 2026-02-19T20:44:29.692 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75331.log 2026-02-19T20:44:29.692 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74980.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80813.log 2026-02-19T20:44:29.692 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51530.log 2026-02-19T20:44:29.692 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.74980.log.gz 2026-02-19T20:44:29.693 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75331.log: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55957.log.gz 2026-02-19T20:44:29.693 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.80813.log: gzip -5 --verbose 87.4% -- /var/log/ceph/cluster1-client.mirror.54866.log 2026-02-19T20:44:29.693 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.75331.log.gz 2026-02-19T20:44:29.693 INFO:teuthology.orchestra.run.trial080.stderr: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.80813.log.gz 2026-02-19T20:44:29.693 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.51530.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51116.log 2026-02-19T20:44:29.693 INFO:teuthology.orchestra.run.trial080.stderr: 87.8% -- replaced with /var/log/ceph/cluster1-client.mirror.51530.log.gz 2026-02-19T20:44:29.693 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54866.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64281.log 2026-02-19T20:44:29.693 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51116.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54935.log 2026-02-19T20:44:29.694 INFO:teuthology.orchestra.run.trial080.stderr: 89.2% -- replaced with /var/log/ceph/cluster1-client.mirror.54866.log.gz 2026-02-19T20:44:29.694 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64281.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58981.log 2026-02-19T20:44:29.694 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.51116.log.gz 2026-02-19T20:44:29.694 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% 87.5%/var/log/ceph/cluster2-client.mirror.54935.log: -- replaced with /var/log/ceph/cluster1-client.mirror.64281.log.gz 2026-02-19T20:44:29.694 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.55505.log.gz 2026-02-19T20:44:29.694 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70531.log 2026-02-19T20:44:29.694 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58981.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71612.log 2026-02-19T20:44:29.695 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70531.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84881.log 2026-02-19T20:44:29.695 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.70531.log.gz 2026-02-19T20:44:29.695 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71612.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76891.log 2026-02-19T20:44:29.695 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84881.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52918.log 2026-02-19T20:44:29.696 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76891.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79207.log 2026-02-19T20:44:29.696 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.76891.log.gz 2026-02-19T20:44:29.696 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52918.log: gzip 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.71612.log.gz 2026-02-19T20:44:29.696 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose 90.7% -- replaced with /var/log/ceph/cluster2-client.mirror.84881.log.gz 2026-02-19T20:44:29.696 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster2-client.mirror.72455.log 2026-02-19T20:44:29.696 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79207.log: 88.4% -- replaced with /var/log/ceph/cluster1-client.mirror.52918.log.gz 2026-02-19T20:44:29.697 INFO:teuthology.orchestra.run.trial080.stderr: 86.6%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79072.log 2026-02-19T20:44:29.697 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.79207.log.gz 2026-02-19T20:44:29.697 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72455.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84621.log 2026-02-19T20:44:29.697 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.72455.log.gz 2026-02-19T20:44:29.697 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.54935.log.gz 2026-02-19T20:44:29.697 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79072.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79765.log 2026-02-19T20:44:29.697 INFO:teuthology.orchestra.run.trial080.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.79072.log.gz 2026-02-19T20:44:29.697 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84621.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61239.log 2026-02-19T20:44:29.697 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84621.log.gz 2026-02-19T20:44:29.698 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79765.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55570.log 2026-02-19T20:44:29.698 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.79765.log.gz 2026-02-19T20:44:29.698 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61239.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72061.log 2026-02-19T20:44:29.698 INFO:teuthology.orchestra.run.trial080.stderr: 92.8% -- replaced with /var/log/ceph/cluster1-client.mirror.0.69756.log.gz 2026-02-19T20:44:29.698 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55570.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68347.log 2026-02-19T20:44:29.699 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72061.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66816.log 2026-02-19T20:44:29.699 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.72061.log.gz 2026-02-19T20:44:29.699 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68347.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64159.log 2026-02-19T20:44:29.699 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66816.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78195.log 2026-02-19T20:44:29.699 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.66816.log.gz 2026-02-19T20:44:29.700 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64159.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81862.log 2026-02-19T20:44:29.700 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78195.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60980.log 2026-02-19T20:44:29.700 INFO:teuthology.orchestra.run.trial080.stderr: 89.0% 92.0% -- replaced with /var/log/ceph/cluster2-client.mirror.64159.log.gz 2026-02-19T20:44:29.700 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.68347.log.gz 2026-02-19T20:44:29.700 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78195.log.gz 2026-02-19T20:44:29.700 INFO:teuthology.orchestra.run.trial080.stderr:gzip/var/log/ceph/cluster1-client.mirror.81862.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56106.log 2026-02-19T20:44:29.700 INFO:teuthology.orchestra.run.trial080.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.81862.log.gz 2026-02-19T20:44:29.700 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60980.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67993.log 2026-02-19T20:44:29.701 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56106.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77118.log 2026-02-19T20:44:29.701 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67993.log: 88.1%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64994.log 2026-02-19T20:44:29.701 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.56106.log.gz 2026-02-19T20:44:29.701 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.67993.log.gz 2026-02-19T20:44:29.702 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.77118.log: gzip 93.6% -- replaced with /var/log/ceph/cluster2-client.mirror.54729.log.gz 2026-02-19T20:44:29.702 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52894.log 2026-02-19T20:44:29.702 INFO:teuthology.orchestra.run.trial080.stderr: 87.3%/var/log/ceph/cluster1-client.mirror.64994.log: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55570.log.gz 2026-02-19T20:44:29.702 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.77118.log.gz 2026-02-19T20:44:29.702 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63711.log 2026-02-19T20:44:29.702 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.64994.log.gz 2026-02-19T20:44:29.702 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52894.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83737.log 2026-02-19T20:44:29.702 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster1-client.mirror.52894.log.gz 2026-02-19T20:44:29.702 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63711.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79551.log 2026-02-19T20:44:29.703 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.63711.log.gz 2026-02-19T20:44:29.703 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59979.log 2026-02-19T20:44:29.703 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83737.log: /var/log/ceph/cluster2-client.mirror.79551.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.83737.log.gz 2026-02-19T20:44:29.703 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63257.log 2026-02-19T20:44:29.703 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59979.log: gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.79551.log.gz 2026-02-19T20:44:29.703 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster2-client.mirror.75930.log 2026-02-19T20:44:29.704 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63257.log: gzip -5 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.63257.log.gz 2026-02-19T20:44:29.704 INFO:teuthology.orchestra.run.trial080.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.73920.log 2026-02-19T20:44:29.704 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75930.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82597.log 2026-02-19T20:44:29.705 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73920.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81945.log 2026-02-19T20:44:29.705 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82597.log: 90.0%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54014.log 2026-02-19T20:44:29.705 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.75930.log.gz 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.82597.log.gz 2026-02-19T20:44:29.705 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:29.705 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81945.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62594.log 2026-02-19T20:44:29.705 INFO:teuthology.orchestra.run.trial080.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.73920.log.gz 2026-02-19T20:44:29.705 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81945.log.gz 2026-02-19T20:44:29.706 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54014.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57503.log 2026-02-19T20:44:29.706 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62594.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57328.log 2026-02-19T20:44:29.707 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57503.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67220.log 2026-02-19T20:44:29.707 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.57503.log.gz 2026-02-19T20:44:29.707 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57328.log: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57328.log.gz 2026-02-19T20:44:29.707 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85034.log 2026-02-19T20:44:29.707 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67220.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.62594.log.gz 2026-02-19T20:44:29.707 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70959.log 2026-02-19T20:44:29.707 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.85034.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.67220.log.gz 2026-02-19T20:44:29.708 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.85034.log.gz 2026-02-19T20:44:29.708 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82027.log 2026-02-19T20:44:29.708 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70959.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75682.log 2026-02-19T20:44:29.708 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82027.log: 88.2%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84662.log 2026-02-19T20:44:29.708 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.70959.log.gz 2026-02-19T20:44:29.709 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.82027.log.gz/var/log/ceph/cluster2-client.mirror.75682.log: 2026-02-19T20:44:29.709 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55980.log 2026-02-19T20:44:29.709 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84662.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.54014.log.gz 2026-02-19T20:44:29.709 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63910.log 2026-02-19T20:44:29.709 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84662.log.gz 2026-02-19T20:44:29.709 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80029.log 2026-02-19T20:44:29.709 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55980.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.75682.log.gz 2026-02-19T20:44:29.710 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63910.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64821.log 2026-02-19T20:44:29.710 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80029.log: gzip 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.63910.log.gz 2026-02-19T20:44:29.710 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79514.log 2026-02-19T20:44:29.711 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64821.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66536.log 2026-02-19T20:44:29.711 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79514.log: gzip -5 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.79514.log.gz 2026-02-19T20:44:29.711 INFO:teuthology.orchestra.run.trial080.stderr: --verbose -- /var/log/ceph/cluster2-client.admin.46608.log 2026-02-19T20:44:29.711 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66536.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73217.log 2026-02-19T20:44:29.711 INFO:teuthology.orchestra.run.trial080.stderr: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.64821.log.gz 2026-02-19T20:44:29.712 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46608.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.66536.log.gz 2026-02-19T20:44:29.712 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46608.log.gz 2026-02-19T20:44:29.712 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75703.log 2026-02-19T20:44:29.712 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73217.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54203.log 2026-02-19T20:44:29.712 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75703.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83757.log 2026-02-19T20:44:29.713 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75703.log.gz 2026-02-19T20:44:29.713 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54203.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55980.log.gz 2026-02-19T20:44:29.713 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.73217.log.gz 2026-02-19T20:44:29.713 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster1-client.mirror.63007.log 2026-02-19T20:44:29.713 INFO:teuthology.orchestra.run.trial080.stderr: 89.6% -- replaced with /var/log/ceph/cluster2-client.mirror.54203.log.gz 2026-02-19T20:44:29.713 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83757.log: gzip -5 --verbose 86.7% -- replaced with /var/log/ceph/cluster2-client.mirror.83757.log.gz 2026-02-19T20:44:29.713 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster1-client.mirror.84535.log 2026-02-19T20:44:29.714 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63007.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81581.log 2026-02-19T20:44:29.714 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.63007.log.gz 2026-02-19T20:44:29.714 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.84535.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64704.log 2026-02-19T20:44:29.714 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.84535.log.gz 2026-02-19T20:44:29.715 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81581.log: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.81581.log.gz 2026-02-19T20:44:29.715 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60032.log 2026-02-19T20:44:29.715 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64704.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63984.log 2026-02-19T20:44:29.715 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.64704.log.gz 2026-02-19T20:44:29.715 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60032.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72355.log 2026-02-19T20:44:29.716 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63984.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60032.log.gz 2026-02-19T20:44:29.716 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.63984.log.gz 2026-02-19T20:44:29.716 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster2-client.mirror.82658.log 2026-02-19T20:44:29.716 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72355.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74773.log 2026-02-19T20:44:29.717 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82658.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84132.log 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.82658.log.gz 2026-02-19T20:44:29.717 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:29.717 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74773.log: gzip -5 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.72355.log.gz 2026-02-19T20:44:29.717 INFO:teuthology.orchestra.run.trial080.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.69869.log 2026-02-19T20:44:29.717 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74773.log.gz 2026-02-19T20:44:29.718 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84132.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72700.log 2026-02-19T20:44:29.718 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84132.log.gz 2026-02-19T20:44:29.718 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69869.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66766.log 2026-02-19T20:44:29.718 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.69869.log.gz 2026-02-19T20:44:29.719 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72700.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79826.log 2026-02-19T20:44:29.719 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster2-client.mirror.72700.log.gz 2026-02-19T20:44:29.719 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66766.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76096.log 2026-02-19T20:44:29.719 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79826.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.66766.log.gz 2026-02-19T20:44:29.719 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83270.log 2026-02-19T20:44:29.719 INFO:teuthology.orchestra.run.trial080.stderr: 86.7% -- replaced with /var/log/ceph/cluster2-client.mirror.79826.log.gz 2026-02-19T20:44:29.720 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76096.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64841.log 2026-02-19T20:44:29.720 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.76096.log.gz 2026-02-19T20:44:29.720 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83270.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78982.log 2026-02-19T20:44:29.720 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64841.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57207.log 2026-02-19T20:44:29.721 INFO:teuthology.orchestra.run.trial080.stderr: 90.7%/var/log/ceph/cluster1-client.mirror.78982.log: -- replaced with /var/log/ceph/cluster2-client.mirror.83270.log.gz 2026-02-19T20:44:29.721 INFO:teuthology.orchestra.run.trial080.stderr:gzip 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.78982.log.gz 2026-02-19T20:44:29.721 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82952.log 2026-02-19T20:44:29.721 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57207.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.64841.log.gz 2026-02-19T20:44:29.721 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62505.log 2026-02-19T20:44:29.722 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57207.log.gz 2026-02-19T20:44:29.722 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82952.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79496.log 2026-02-19T20:44:29.722 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.82952.log.gz 2026-02-19T20:44:29.722 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62505.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78172.log 2026-02-19T20:44:29.722 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62505.log.gz 2026-02-19T20:44:29.722 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79496.log: 41.0% -- replaced with /var/log/ceph/cluster2-client.mirror.79496.log.gz 2026-02-19T20:44:29.723 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52233.log 2026-02-19T20:44:29.723 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78172.log: gzip -5 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78172.log.gz 2026-02-19T20:44:29.723 INFO:teuthology.orchestra.run.trial080.stderr: --verbose -- /var/log/ceph/cluster2-client.mirror.57523.log 2026-02-19T20:44:29.724 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52233.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67543.log 2026-02-19T20:44:29.724 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57523.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58668.log 2026-02-19T20:44:29.724 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57523.log.gz 2026-02-19T20:44:29.724 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67543.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50914.log 2026-02-19T20:44:29.724 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.67543.log.gz 2026-02-19T20:44:29.725 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58668.log: gzip -5 --verbose -- 87.2% /var/log/ceph/cluster1-client.mirror.78487.log 2026-02-19T20:44:29.725 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.58668.log.gz 2026-02-19T20:44:29.725 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50914.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57856.log 2026-02-19T20:44:29.725 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.50914.log.gz 2026-02-19T20:44:29.725 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78487.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50938.log 2026-02-19T20:44:29.726 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78487.log.gz 2026-02-19T20:44:29.726 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57856.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55086.log 2026-02-19T20:44:29.726 INFO:teuthology.orchestra.run.trial080.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51658.log 2026-02-19T20:44:29.726 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50938.log: -- replaced with /var/log/ceph/cluster2-client.mirror.57856.log.gz 2026-02-19T20:44:29.726 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55086.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.50938.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58093.log 2026-02-19T20:44:29.726 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:29.727 INFO:teuthology.orchestra.run.trial080.stderr: 89.1% -- replaced with /var/log/ceph/cluster2-client.mirror.55086.log.gz 2026-02-19T20:44:29.727 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51658.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61217.log 2026-02-19T20:44:29.727 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58093.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61069.log 2026-02-19T20:44:29.727 INFO:teuthology.orchestra.run.trial080.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58093.log.gz 2026-02-19T20:44:29.728 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61217.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61217.log.gz 2026-02-19T20:44:29.728 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster2-client.admin.43469.log 2026-02-19T20:44:29.728 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61069.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.1.81684.log 2026-02-19T20:44:29.728 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.43469.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.43469.log.gz 2026-02-19T20:44:29.728 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74339.log 2026-02-19T20:44:29.729 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.1.81684.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56897.log 2026-02-19T20:44:29.729 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74339.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59354.log 2026-02-19T20:44:29.729 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74339.log.gz 2026-02-19T20:44:29.730 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56897.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80658.log 2026-02-19T20:44:29.730 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59354.log: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.59354.log.gz 2026-02-19T20:44:29.730 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57049.log 2026-02-19T20:44:29.731 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80658.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.80658.log.gz 2026-02-19T20:44:29.731 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53663.log 2026-02-19T20:44:29.732 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57049.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57049.log.gz 2026-02-19T20:44:29.732 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.1.81684.log.gz 2026-02-19T20:44:29.732 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75124.log 2026-02-19T20:44:29.733 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53663.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80094.log 2026-02-19T20:44:29.734 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75124.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.75124.log.gz 2026-02-19T20:44:29.736 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.53663.log.gz 2026-02-19T20:44:29.736 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-osd.0.log 2026-02-19T20:44:29.736 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80094.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54998.log 2026-02-19T20:44:29.737 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-osd.0.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.80094.log.gz 2026-02-19T20:44:29.737 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78712.log 2026-02-19T20:44:29.740 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54998.log: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.54998.log.gz 2026-02-19T20:44:29.741 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71053.log 2026-02-19T20:44:29.741 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78712.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78712.log.gz 2026-02-19T20:44:29.741 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47637.log 2026-02-19T20:44:29.742 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71053.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.71053.log.gz 2026-02-19T20:44:29.742 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57935.log 2026-02-19T20:44:29.742 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47637.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47637.log.gz 2026-02-19T20:44:29.743 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81618.log 2026-02-19T20:44:29.743 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57935.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.57935.log.gz 2026-02-19T20:44:29.744 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.43383.log 2026-02-19T20:44:29.744 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81618.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.81618.log.gz 2026-02-19T20:44:29.745 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47995.log 2026-02-19T20:44:29.745 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.43383.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.43383.log.gz 2026-02-19T20:44:29.745 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66019.log 2026-02-19T20:44:29.746 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.47995.log: 74.8% -- replaced with /var/log/ceph/cluster2-client.mirror.47995.log.gz 2026-02-19T20:44:29.746 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50534.log 2026-02-19T20:44:29.747 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66019.log: 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.66019.log.gz 2026-02-19T20:44:29.748 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67061.log 2026-02-19T20:44:29.748 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50534.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.50534.log.gz 2026-02-19T20:44:29.748 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68764.log 2026-02-19T20:44:29.749 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67061.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.67061.log.gz 2026-02-19T20:44:29.749 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73558.log 2026-02-19T20:44:29.750 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68764.log: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.68764.log.gz 2026-02-19T20:44:29.750 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71469.log 2026-02-19T20:44:29.751 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73558.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.73558.log.gz 2026-02-19T20:44:29.751 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61830.log 2026-02-19T20:44:29.752 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71469.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.71469.log.gz 2026-02-19T20:44:29.752 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75352.log 2026-02-19T20:44:29.753 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61830.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.61830.log.gz 2026-02-19T20:44:29.753 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65677.log 2026-02-19T20:44:29.753 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75352.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75352.log.gz 2026-02-19T20:44:29.754 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.85055.log 2026-02-19T20:44:29.754 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65677.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65677.log.gz 2026-02-19T20:44:29.755 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53807.log 2026-02-19T20:44:29.755 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.85055.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.85055.log.gz 2026-02-19T20:44:29.755 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62047.log 2026-02-19T20:44:29.756 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53807.log: 89.5% -- replaced with /var/log/ceph/cluster2-client.mirror.53807.log.gz 2026-02-19T20:44:29.756 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60752.log 2026-02-19T20:44:29.757 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62047.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62047.log.gz 2026-02-19T20:44:29.757 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65105.log 2026-02-19T20:44:29.758 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60752.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60752.log.gz 2026-02-19T20:44:29.758 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77790.log 2026-02-19T20:44:29.759 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65105.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.65105.log.gz 2026-02-19T20:44:29.759 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61027.log 2026-02-19T20:44:29.759 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77790.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77790.log.gz 2026-02-19T20:44:29.760 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74711.log 2026-02-19T20:44:29.760 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61027.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61027.log.gz 2026-02-19T20:44:29.761 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.53247.log 2026-02-19T20:44:29.761 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74711.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.74711.log.gz 2026-02-19T20:44:29.761 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53390.log 2026-02-19T20:44:29.762 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.53247.log: 87.7% -- replaced with /var/log/ceph/cluster1-client.admin.53247.log.gz 2026-02-19T20:44:29.763 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81021.log 2026-02-19T20:44:29.763 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53390.log: 88.4% -- replaced with /var/log/ceph/cluster1-client.mirror.53390.log.gz 2026-02-19T20:44:29.764 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58922.log 2026-02-19T20:44:29.764 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81021.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81021.log.gz 2026-02-19T20:44:29.765 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.47937.log 2026-02-19T20:44:29.765 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58922.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58922.log.gz 2026-02-19T20:44:29.766 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64119.log 2026-02-19T20:44:29.772 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.2.47937.log: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.3.51850.log.gz 2026-02-19T20:44:29.773 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68413.log 2026-02-19T20:44:29.773 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64119.log: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.64119.log.gz 2026-02-19T20:44:29.774 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81640.log 2026-02-19T20:44:29.775 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68413.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68413.log.gz 2026-02-19T20:44:29.775 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55290.log 2026-02-19T20:44:29.789 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81640.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80939.log 2026-02-19T20:44:29.793 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55290.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55290.log.gz 2026-02-19T20:44:29.796 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48515.log 2026-02-19T20:44:29.797 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80939.log: 86.9% -- replaced with /var/log/ceph/cluster2-client.mirror.80939.log.gz 2026-02-19T20:44:29.809 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51636.log 2026-02-19T20:44:29.810 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48515.log: 3.5% -- replaced with /var/log/ceph/cluster1-client.mirror.48515.log.gz 2026-02-19T20:44:29.822 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55679.log 2026-02-19T20:44:29.823 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51636.log: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.51636.log.gz 2026-02-19T20:44:29.831 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83593.log 2026-02-19T20:44:29.835 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55679.log: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.55679.log.gz 2026-02-19T20:44:29.851 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58760.log 2026-02-19T20:44:29.851 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83593.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.83593.log.gz 2026-02-19T20:44:29.858 INFO:teuthology.orchestra.run.trial080.stderr: 94.1% -- replaced with /var/log/ceph/cluster1-client.mirror.2.47937.log.gz 2026-02-19T20:44:29.858 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58611.log 2026-02-19T20:44:29.859 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58760.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58760.log.gz 2026-02-19T20:44:29.859 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55439.log 2026-02-19T20:44:29.864 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58611.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58880.log.gz 2026-02-19T20:44:29.870 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81684.log 2026-02-19T20:44:29.874 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55439.log: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55439.log.gz 2026-02-19T20:44:29.874 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56875.log 2026-02-19T20:44:29.875 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.81684.log: 86.8% -- replaced with /var/log/ceph/cluster1-client.mirror.81684.log.gz 2026-02-19T20:44:29.875 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58589.log 2026-02-19T20:44:29.876 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56875.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.56875.log.gz 2026-02-19T20:44:29.876 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76746.log 2026-02-19T20:44:29.877 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58589.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58589.log.gz 2026-02-19T20:44:29.877 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76034.log 2026-02-19T20:44:29.878 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76746.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76746.log.gz 2026-02-19T20:44:29.878 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59784.log 2026-02-19T20:44:29.879 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76034.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.76034.log.gz 2026-02-19T20:44:29.879 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61629.log 2026-02-19T20:44:29.880 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59784.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.59784.log.gz 2026-02-19T20:44:29.881 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67178.log 2026-02-19T20:44:29.881 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61629.log: 1.3% -- replaced with /var/log/ceph/cluster1-client.mirror.61629.log.gz 2026-02-19T20:44:29.881 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71808.log 2026-02-19T20:44:29.882 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67178.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.67178.log.gz 2026-02-19T20:44:29.883 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82867.log 2026-02-19T20:44:29.883 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71808.log: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.71808.log.gz 2026-02-19T20:44:29.884 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57249.log 2026-02-19T20:44:29.885 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.82867.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.82867.log.gz 2026-02-19T20:44:29.885 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66721.log 2026-02-19T20:44:29.886 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57249.log: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.57249.log.gz 2026-02-19T20:44:29.886 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54890.log 2026-02-19T20:44:29.886 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66721.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.66721.log.gz 2026-02-19T20:44:29.887 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61893.log 2026-02-19T20:44:29.888 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54890.log: 89.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54890.log.gz 2026-02-19T20:44:29.888 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70692.log 2026-02-19T20:44:29.889 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61893.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.61893.log.gz 2026-02-19T20:44:29.889 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68184.log 2026-02-19T20:44:29.891 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70692.log: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.70692.log.gz 2026-02-19T20:44:29.891 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77035.log 2026-02-19T20:44:29.892 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68184.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68184.log.gz 2026-02-19T20:44:29.892 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61587.log 2026-02-19T20:44:29.893 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.77035.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.77035.log.gz 2026-02-19T20:44:29.894 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71654.log 2026-02-19T20:44:29.894 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61587.log: 1.3% -- replaced with /var/log/ceph/cluster1-client.mirror.61587.log.gz 2026-02-19T20:44:29.895 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85378.log 2026-02-19T20:44:29.895 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71654.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.71654.log.gz 2026-02-19T20:44:29.896 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82151.log 2026-02-19T20:44:29.896 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85378.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85378.log.gz 2026-02-19T20:44:29.897 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62806.log 2026-02-19T20:44:29.897 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82151.log: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.82151.log.gz 2026-02-19T20:44:29.899 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67820.log 2026-02-19T20:44:29.899 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62806.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62806.log.gz 2026-02-19T20:44:29.900 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69479.log 2026-02-19T20:44:29.900 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67820.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.67820.log.gz 2026-02-19T20:44:29.901 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50510.log 2026-02-19T20:44:29.901 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69479.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.69479.log.gz 2026-02-19T20:44:29.902 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83920.log 2026-02-19T20:44:29.902 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50510.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.50510.log.gz 2026-02-19T20:44:29.903 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73582.log 2026-02-19T20:44:29.911 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83920.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57345.log 2026-02-19T20:44:29.925 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73582.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.73582.log.gz 2026-02-19T20:44:29.926 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48549.log 2026-02-19T20:44:29.926 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57345.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57345.log.gz 2026-02-19T20:44:29.935 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52033.log 2026-02-19T20:44:29.947 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48549.log: 2.3% -- replaced with /var/log/ceph/cluster1-client.mirror.48549.log.gz 2026-02-19T20:44:29.955 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63281.log 2026-02-19T20:44:29.956 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52033.log: 87.8% -- replaced with /var/log/ceph/cluster1-client.mirror.52033.log.gz 2026-02-19T20:44:29.964 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73803.log 2026-02-19T20:44:29.977 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63281.log: 87.6% 93.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58690.log.gz 2026-02-19T20:44:29.977 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.63281.log.gz 2026-02-19T20:44:29.977 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51434.log 2026-02-19T20:44:29.978 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73803.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.73803.log.gz 2026-02-19T20:44:29.978 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.43685.log 2026-02-19T20:44:29.979 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.51434.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.51434.log.gz 2026-02-19T20:44:29.979 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56925.log 2026-02-19T20:44:29.980 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.43685.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.43685.log.gz 2026-02-19T20:44:29.980 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78510.log 2026-02-19T20:44:29.981 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56925.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.56925.log.gz 2026-02-19T20:44:29.981 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67037.log 2026-02-19T20:44:29.982 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78510.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.78510.log.gz 2026-02-19T20:44:29.982 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72797.log 2026-02-19T20:44:29.983 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67037.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.67037.log.gz 2026-02-19T20:44:29.983 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76178.log 2026-02-19T20:44:29.984 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72797.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.72797.log.gz 2026-02-19T20:44:29.984 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58331.log 2026-02-19T20:44:29.985 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76178.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76178.log.gz 2026-02-19T20:44:29.986 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68234.log 2026-02-19T20:44:29.986 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58331.log: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.58331.log.gz 2026-02-19T20:44:29.987 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77077.log 2026-02-19T20:44:29.988 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68234.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68234.log.gz 2026-02-19T20:44:29.988 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78825.log 2026-02-19T20:44:29.989 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.77077.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.77077.log.gz 2026-02-19T20:44:29.989 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69636.log 2026-02-19T20:44:29.990 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78825.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78825.log.gz 2026-02-19T20:44:29.990 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67943.log 2026-02-19T20:44:29.991 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69636.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.69636.log.gz 2026-02-19T20:44:29.991 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72334.log 2026-02-19T20:44:29.992 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67943.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.67943.log.gz 2026-02-19T20:44:29.992 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84432.log 2026-02-19T20:44:29.993 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72334.log: 87.7% -- replaced with /var/log/ceph/cluster1-client.mirror.72334.log.gz 2026-02-19T20:44:29.993 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.53052.log 2026-02-19T20:44:29.994 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84432.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.84432.log.gz 2026-02-19T20:44:29.994 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79908.log 2026-02-19T20:44:29.999 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.0.53052.log: 93.2% -- replaced with /var/log/ceph/cluster2-client.mirror.0.53052.log.gz 2026-02-19T20:44:29.999 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82932.log 2026-02-19T20:44:30.000 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79908.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.79908.log.gz 2026-02-19T20:44:30.000 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82210.log 2026-02-19T20:44:30.001 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82932.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.82932.log.gz 2026-02-19T20:44:30.001 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72148.log 2026-02-19T20:44:30.010 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82210.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70221.log 2026-02-19T20:44:30.024 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72148.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.72148.log.gz 2026-02-19T20:44:30.026 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57738.log 2026-02-19T20:44:30.027 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70221.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.70221.log.gz 2026-02-19T20:44:30.032 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.80726.log.gz 2026-02-19T20:44:30.033 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52942.log 2026-02-19T20:44:30.041 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57738.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70022.log 2026-02-19T20:44:30.054 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52942.log: 88.7%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57545.log 2026-02-19T20:44:30.054 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.52942.log.gz 2026-02-19T20:44:30.055 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70022.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.70022.log.gz 2026-02-19T20:44:30.068 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56661.log 2026-02-19T20:44:30.075 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57545.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51850.log 2026-02-19T20:44:30.082 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56661.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53993.log 2026-02-19T20:44:30.084 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51850.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54121.log.gz 2026-02-19T20:44:30.102 INFO:teuthology.orchestra.run.trial080.stderr: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.51658.log.gz 2026-02-19T20:44:30.102 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65063.log 2026-02-19T20:44:30.104 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose --/var/log/ceph/cluster1-client.mirror.53993.log: /var/log/ceph/cluster2-client.mirror.71895.log 2026-02-19T20:44:30.104 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster1-client.mirror.53993.log.gz 2026-02-19T20:44:30.106 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65063.log: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.65063.log.gz 2026-02-19T20:44:30.106 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74566.log 2026-02-19T20:44:30.106 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71895.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.71895.log.gz 2026-02-19T20:44:30.106 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69910.log 2026-02-19T20:44:30.108 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74566.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74566.log.gz 2026-02-19T20:44:30.108 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64077.log 2026-02-19T20:44:30.108 INFO:teuthology.orchestra.run.trial073.stderr: 90.1% -- replaced with /var/log/ceph/cluster1-mon.a.log.gz 2026-02-19T20:44:30.110 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69910.log: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.69910.log.gz 2026-02-19T20:44:30.110 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70603.log 2026-02-19T20:44:30.111 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64077.log: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.64077.log.gz 2026-02-19T20:44:30.112 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59920.log 2026-02-19T20:44:30.115 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70603.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.70603.log.gz 2026-02-19T20:44:30.115 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63617.log 2026-02-19T20:44:30.115 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59920.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.59920.log.gz 2026-02-19T20:44:30.116 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61313.log 2026-02-19T20:44:30.116 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63617.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.63617.log.gz 2026-02-19T20:44:30.116 INFO:teuthology.orchestra.run.trial080.stderr:gzip 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.61239.log.gz 2026-02-19T20:44:30.116 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59413.log 2026-02-19T20:44:30.116 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75228.log 2026-02-19T20:44:30.116 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61313.log: 87.6%/var/log/ceph/cluster2-client.mirror.59413.log: -- replaced with /var/log/ceph/cluster2-client.mirror.61313.log.gz 2026-02-19T20:44:30.119 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.81184.log.gz 2026-02-19T20:44:30.124 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70555.log 2026-02-19T20:44:30.124 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75228.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.75228.log.gz 2026-02-19T20:44:30.129 INFO:teuthology.orchestra.run.trial080.stderr: 93.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56661.log.gz 2026-02-19T20:44:30.133 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59021.log 2026-02-19T20:44:30.133 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70555.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.70555.log.gz 2026-02-19T20:44:30.133 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61134.log 2026-02-19T20:44:30.133 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59021.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59021.log.gz 2026-02-19T20:44:30.133 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62686.log 2026-02-19T20:44:30.133 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61134.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.61134.log.gz 2026-02-19T20:44:30.133 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65725.log 2026-02-19T20:44:30.133 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62686.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.62686.log.gz 2026-02-19T20:44:30.134 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53945.log 2026-02-19T20:44:30.137 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65725.log: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.65725.log.gz 2026-02-19T20:44:30.138 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51959.log 2026-02-19T20:44:30.138 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53945.log: 88.8% -- replaced with /var/log/ceph/cluster1-client.mirror.53945.log.gz 2026-02-19T20:44:30.138 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73827.log 2026-02-19T20:44:30.138 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51959.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.51959.log.gz 2026-02-19T20:44:30.138 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68622.log 2026-02-19T20:44:30.138 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73827.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.73827.log.gz 2026-02-19T20:44:30.138 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84370.log 2026-02-19T20:44:30.138 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68622.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.68622.log.gz 2026-02-19T20:44:30.143 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46808.log 2026-02-19T20:44:30.143 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84370.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84370.log.gz 2026-02-19T20:44:30.143 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64728.log 2026-02-19T20:44:30.143 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46808.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46808.log.gz 2026-02-19T20:44:30.143 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80492.log 2026-02-19T20:44:30.143 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64728.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.64728.log.gz 2026-02-19T20:44:30.143 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79410.log 2026-02-19T20:44:30.143 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80492.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.80492.log.gz 2026-02-19T20:44:30.143 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52846.log 2026-02-19T20:44:30.144 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79410.log: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.79410.log.gz 2026-02-19T20:44:30.144 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66151.log 2026-02-19T20:44:30.147 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52846.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.52846.log.gz 2026-02-19T20:44:30.147 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79806.log 2026-02-19T20:44:30.147 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66151.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.66151.log.gz 2026-02-19T20:44:30.147 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65191.log 2026-02-19T20:44:30.147 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79806.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.79806.log.gz 2026-02-19T20:44:30.147 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79365.log 2026-02-19T20:44:30.147 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65191.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65191.log.gz 2026-02-19T20:44:30.147 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65856.log 2026-02-19T20:44:30.153 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79365.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.79365.log.gz 2026-02-19T20:44:30.153 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84350.log 2026-02-19T20:44:30.153 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65856.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.65856.log.gz 2026-02-19T20:44:30.153 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66392.log 2026-02-19T20:44:30.153 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84350.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58294.log.gz 2026-02-19T20:44:30.153 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.84350.log.gz 2026-02-19T20:44:30.153 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57797.log 2026-02-19T20:44:30.153 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66392.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56066.log 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.66392.log.gz 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57797.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.57797.log.gz 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68716.log 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47588.log 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56066.log: /var/log/ceph/cluster1-client.mirror.68716.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56066.log.gz 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.68716.log.gz 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78285.log 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47937.log 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47588.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47588.log.gz 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78285.log: 86.4%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66933.log 2026-02-19T20:44:30.154 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.78285.log.gz 2026-02-19T20:44:30.164 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.47937.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66699.log 2026-02-19T20:44:30.165 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66933.log: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.66933.log.gz 2026-02-19T20:44:30.177 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82800.log 2026-02-19T20:44:30.178 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66699.log: 94.0% 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.66699.log.gz 2026-02-19T20:44:30.178 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.51850.log.gz 2026-02-19T20:44:30.178 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49010.log 2026-02-19T20:44:30.180 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.82800.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48600.log 2026-02-19T20:44:30.180 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.82800.log.gz 2026-02-19T20:44:30.181 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49010.log: 62.9% -- replaced with /var/log/ceph/cluster2-client.mirror.49010.log.gz 2026-02-19T20:44:30.181 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57266.log 2026-02-19T20:44:30.181 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48600.log: 63.8%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75248.log 2026-02-19T20:44:30.181 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.48600.log.gz 2026-02-19T20:44:30.181 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57266.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65018.log 2026-02-19T20:44:30.181 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75248.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.57266.log.gz 2026-02-19T20:44:30.187 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81458.log 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65018.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.75248.log.gz 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.65018.log.gz 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63661.log 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81458.log: gzip -5 --verbose -- /var/log/ceph/cluster2-osd.1.log 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.81458.log.gz 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63661.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64654.log 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-osd.1.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.63661.log.gz 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76932.log 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64654.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.64654.log.gz 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68985.log 2026-02-19T20:44:30.188 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76932.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.76932.log.gz 2026-02-19T20:44:30.189 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63097.log 2026-02-19T20:44:30.189 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68985.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68985.log.gz 2026-02-19T20:44:30.189 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72772.log 2026-02-19T20:44:30.191 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63097.log: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.63097.log.gz 2026-02-19T20:44:30.191 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68110.log 2026-02-19T20:44:30.191 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72772.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.72772.log.gz 2026-02-19T20:44:30.191 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61851.log 2026-02-19T20:44:30.194 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68110.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.68110.log.gz 2026-02-19T20:44:30.194 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53007.log 2026-02-19T20:44:30.194 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61851.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61851.log.gz 2026-02-19T20:44:30.194 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51031.log 2026-02-19T20:44:30.198 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53007.log: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.53007.log.gz 2026-02-19T20:44:30.198 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62204.log 2026-02-19T20:44:30.201 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51031.log: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.51031.log.gz 2026-02-19T20:44:30.202 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58453.log 2026-02-19T20:44:30.202 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62204.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.62204.log.gz 2026-02-19T20:44:30.203 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71760.log 2026-02-19T20:44:30.212 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58453.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54588.log 2026-02-19T20:44:30.222 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71760.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.61424.log.gz 2026-02-19T20:44:30.223 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.71760.log.gz 2026-02-19T20:44:30.223 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73695.log 2026-02-19T20:44:30.224 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.54588.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.admin.54588.log.gz 2026-02-19T20:44:30.225 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79929.log 2026-02-19T20:44:30.226 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73695.log: 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.73695.log.gz 2026-02-19T20:44:30.226 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74690.log 2026-02-19T20:44:30.227 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79929.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.79929.log.gz 2026-02-19T20:44:30.227 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82047.log 2026-02-19T20:44:30.228 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74690.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74690.log.gz 2026-02-19T20:44:30.229 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64752.log 2026-02-19T20:44:30.230 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82047.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.82047.log.gz 2026-02-19T20:44:30.230 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70714.log 2026-02-19T20:44:30.231 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64752.log: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.64752.log.gz 2026-02-19T20:44:30.231 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58173.log 2026-02-19T20:44:30.232 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70714.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.70714.log.gz 2026-02-19T20:44:30.232 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80348.log 2026-02-19T20:44:30.233 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58173.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58173.log.gz 2026-02-19T20:44:30.233 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57466.log 2026-02-19T20:44:30.234 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80348.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.80348.log.gz 2026-02-19T20:44:30.234 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74897.log 2026-02-19T20:44:30.244 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57466.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58431.log 2026-02-19T20:44:30.245 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74897.log: 87.4% 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59979.log.gz 2026-02-19T20:44:30.246 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.74897.log.gz 2026-02-19T20:44:30.251 INFO:teuthology.orchestra.run.trial080.stderr: 94.1% -- replaced with /var/log/ceph/cluster2-client.mirror.47937.log.gz 2026-02-19T20:44:30.251 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65545.log 2026-02-19T20:44:30.252 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58431.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66634.log 2026-02-19T20:44:30.252 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58431.log.gz 2026-02-19T20:44:30.253 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65545.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.65545.log.gz 2026-02-19T20:44:30.253 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81125.log 2026-02-19T20:44:30.254 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56574.log 2026-02-19T20:44:30.254 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66634.log: /var/log/ceph/cluster2-client.mirror.81125.log: 87.4% 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.81125.log.gz 2026-02-19T20:44:30.254 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.66634.log.gz 2026-02-19T20:44:30.254 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.80029.log.gz 2026-02-19T20:44:30.255 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63759.log 2026-02-19T20:44:30.255 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56574.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68692.log 2026-02-19T20:44:30.255 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63759.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56574.log.gz 2026-02-19T20:44:30.255 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53438.log 2026-02-19T20:44:30.256 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.63759.log.gz 2026-02-19T20:44:30.256 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70750.log 2026-02-19T20:44:30.256 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68692.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67011.log 2026-02-19T20:44:30.256 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53438.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.68692.log.gz 2026-02-19T20:44:30.256 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster1-client.mirror.53438.log.gz/var/log/ceph/cluster2-client.mirror.70750.log: 2026-02-19T20:44:30.257 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52870.log 2026-02-19T20:44:30.257 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67011.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72527.log 2026-02-19T20:44:30.257 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.70750.log.gz 2026-02-19T20:44:30.257 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52870.log: 88.8% 88.6% -- replaced with /var/log/ceph/cluster1-client.mirror.52870.log.gz 2026-02-19T20:44:30.257 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.67011.log.gz 2026-02-19T20:44:30.258 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76973.log 2026-02-19T20:44:30.258 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75434.log 2026-02-19T20:44:30.258 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72527.log: /var/log/ceph/cluster2-client.mirror.76973.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84173.log 2026-02-19T20:44:30.258 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.72527.log.gz 2026-02-19T20:44:30.259 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75434.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59806.log 2026-02-19T20:44:30.259 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76973.log.gz 2026-02-19T20:44:30.259 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84173.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72238.log 2026-02-19T20:44:30.259 INFO:teuthology.orchestra.run.trial080.stderr: 86.9% -- replaced with /var/log/ceph/cluster2-client.mirror.84173.log.gz 2026-02-19T20:44:30.260 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59806.log: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.75434.log.gz 2026-02-19T20:44:30.260 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58215.log 2026-02-19T20:44:30.260 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72238.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58252.log 2026-02-19T20:44:30.260 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.72238.log.gz 2026-02-19T20:44:30.261 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58215.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64032.log 2026-02-19T20:44:30.262 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58252.log: 88.6% 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58135.log.gz 2026-02-19T20:44:30.262 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.58252.log.gz 2026-02-19T20:44:30.262 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66441.log 2026-02-19T20:44:30.262 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64032.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63687.log 2026-02-19T20:44:30.263 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.64032.log.gz 2026-02-19T20:44:30.263 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66441.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81396.log 2026-02-19T20:44:30.263 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63687.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.66441.log.gz 2026-02-19T20:44:30.263 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.63687.log.gz 2026-02-19T20:44:30.264 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54077.log 2026-02-19T20:44:30.264 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81396.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75600.log 2026-02-19T20:44:30.265 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81396.log.gz 2026-02-19T20:44:30.265 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54077.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76808.log 2026-02-19T20:44:30.265 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75600.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75600.log.gz 2026-02-19T20:44:30.266 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69566.log 2026-02-19T20:44:30.266 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76808.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.76808.log.gz 2026-02-19T20:44:30.267 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77767.log 2026-02-19T20:44:30.267 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69566.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.69566.log.gz 2026-02-19T20:44:30.268 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62660.log 2026-02-19T20:44:30.268 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77767.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77767.log.gz 2026-02-19T20:44:30.269 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74876.log 2026-02-19T20:44:30.269 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62660.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.62660.log.gz 2026-02-19T20:44:30.269 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.54077.log.gz 2026-02-19T20:44:30.270 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52431.log 2026-02-19T20:44:30.270 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74876.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64209.log 2026-02-19T20:44:30.270 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52431.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.52431.log.gz 2026-02-19T20:44:30.271 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.74876.log.gz 2026-02-19T20:44:30.271 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64459.log 2026-02-19T20:44:30.271 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48498.log 2026-02-19T20:44:30.272 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64209.log: /var/log/ceph/cluster1-client.mirror.64459.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.64209.log.gz 2026-02-19T20:44:30.272 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.64459.log.gz 2026-02-19T20:44:30.272 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78847.log 2026-02-19T20:44:30.272 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61545.log 2026-02-19T20:44:30.273 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48498.log: 62.5% -- replaced with /var/log/ceph/cluster1-client.mirror.48498.log.gz 2026-02-19T20:44:30.273 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78847.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60854.log 2026-02-19T20:44:30.273 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61545.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78847.log.gz 2026-02-19T20:44:30.274 INFO:teuthology.orchestra.run.trial080.stderr: 1.3% -- replaced with /var/log/ceph/cluster2-client.mirror.61545.log.gz 2026-02-19T20:44:30.274 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46955.log 2026-02-19T20:44:30.274 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60854.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77587.log 2026-02-19T20:44:30.275 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60854.log.gz 2026-02-19T20:44:30.275 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46955.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46955.log.gz 2026-02-19T20:44:30.275 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75579.log 2026-02-19T20:44:30.275 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77587.log: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.77587.log.gz 2026-02-19T20:44:30.276 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77362.log 2026-02-19T20:44:30.276 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75579.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83655.log 2026-02-19T20:44:30.277 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.75579.log.gz 2026-02-19T20:44:30.277 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77362.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68487.log 2026-02-19T20:44:30.277 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83655.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.77362.log.gz 2026-02-19T20:44:30.277 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.83655.log.gz 2026-02-19T20:44:30.278 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65930.log 2026-02-19T20:44:30.278 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68487.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71407.log 2026-02-19T20:44:30.278 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.68487.log.gz 2026-02-19T20:44:30.279 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65930.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72503.log 2026-02-19T20:44:30.279 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71407.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.65930.log.gz 2026-02-19T20:44:30.280 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.71407.log.gz 2026-02-19T20:44:30.281 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56023.log 2026-02-19T20:44:30.281 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72503.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56809.log 2026-02-19T20:44:30.281 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.72503.log.gz 2026-02-19T20:44:30.282 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56023.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68130.log 2026-02-19T20:44:30.285 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56809.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.56023.log.gz 2026-02-19T20:44:30.285 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64097.log 2026-02-19T20:44:30.286 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68130.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.68130.log.gz 2026-02-19T20:44:30.287 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73675.log 2026-02-19T20:44:30.288 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64097.log: 92.0% -- replaced with /var/log/ceph/cluster1-client.mirror.64097.log.gz 2026-02-19T20:44:30.289 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72917.log 2026-02-19T20:44:30.290 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73675.log: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.73675.log.gz 2026-02-19T20:44:30.291 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74153.log 2026-02-19T20:44:30.291 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72917.log: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.72917.log.gz 2026-02-19T20:44:30.292 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80241.log 2026-02-19T20:44:30.292 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74153.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.74153.log.gz 2026-02-19T20:44:30.293 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57696.log 2026-02-19T20:44:30.293 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.80241.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.80241.log.gz 2026-02-19T20:44:30.294 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82344.log 2026-02-19T20:44:30.295 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57696.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.57696.log.gz 2026-02-19T20:44:30.295 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.81640.log.gz 2026-02-19T20:44:30.295 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65456.log 2026-02-19T20:44:30.295 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.82344.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.51780.log 2026-02-19T20:44:30.296 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.82344.log.gz/var/log/ceph/cluster1-client.mirror.65456.log: 2026-02-19T20:44:30.296 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65456.log.gz 2026-02-19T20:44:30.296 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59667.log 2026-02-19T20:44:30.297 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.51780.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52552.log 2026-02-19T20:44:30.297 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.admin.51780.log.gz 2026-02-19T20:44:30.297 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.82825.log 2026-02-19T20:44:30.298 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59667.log: /var/log/ceph/cluster1-client.mirror.52552.log: 87.9% -- replaced with /var/log/ceph/cluster1-client.mirror.52552.log.gz 2026-02-19T20:44:30.298 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59667.log.gz 2026-02-19T20:44:30.298 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62481.log 2026-02-19T20:44:30.298 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.82825.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.82825.log.gz 2026-02-19T20:44:30.298 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.51290.log 2026-02-19T20:44:30.299 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84469.log 2026-02-19T20:44:30.299 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.51290.log: /var/log/ceph/cluster1-client.mirror.62481.log: 87.2% 87.2% -- replaced with /var/log/ceph/cluster1-client.admin.51290.log.gz 2026-02-19T20:44:30.299 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.62481.log.gz 2026-02-19T20:44:30.300 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60685.log 2026-02-19T20:44:30.300 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84469.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.84469.log.gz 2026-02-19T20:44:30.300 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66562.log 2026-02-19T20:44:30.301 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67885.log 2026-02-19T20:44:30.301 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60685.log: /var/log/ceph/cluster2-client.mirror.66562.log: 87.1% -- replaced with /var/log/ceph/cluster2-client.mirror.66562.log.gz 2026-02-19T20:44:30.302 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.48993.log 2026-02-19T20:44:30.303 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67885.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.67885.log.gz 2026-02-19T20:44:30.303 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69998.log 2026-02-19T20:44:30.304 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.48993.log: 70.6% -- replaced with /var/log/ceph/cluster2-client.admin.48993.log.gz 2026-02-19T20:44:30.304 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83675.log 2026-02-19T20:44:30.305 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69998.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.69998.log.gz 2026-02-19T20:44:30.305 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81249.log 2026-02-19T20:44:30.306 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83675.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.83675.log.gz 2026-02-19T20:44:30.306 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54955.log 2026-02-19T20:44:30.307 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81249.log: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.81249.log.gz 2026-02-19T20:44:30.308 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76261.log 2026-02-19T20:44:30.311 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54955.log: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.54955.log.gz 2026-02-19T20:44:30.311 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82088.log 2026-02-19T20:44:30.312 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76261.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76261.log.gz 2026-02-19T20:44:30.317 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58981.log.gz 2026-02-19T20:44:30.318 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69098.log 2026-02-19T20:44:30.318 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82088.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78645.log 2026-02-19T20:44:30.318 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.82088.log.gz 2026-02-19T20:44:30.319 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69098.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83229.log 2026-02-19T20:44:30.319 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78645.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78645.log.gz 2026-02-19T20:44:30.319 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65504.log 2026-02-19T20:44:30.319 INFO:teuthology.orchestra.run.trial080.stderr: 92.0% -- replaced with /var/log/ceph/cluster1-client.mirror.69098.log.gz 2026-02-19T20:44:30.320 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83229.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.83229.log.gz 2026-02-19T20:44:30.329 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84328.log 2026-02-19T20:44:30.329 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65504.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61473.log 2026-02-19T20:44:30.329 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65504.log.gz 2026-02-19T20:44:30.330 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84328.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82068.log 2026-02-19T20:44:30.330 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61473.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84328.log.gz 2026-02-19T20:44:30.330 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61473.log.gz 2026-02-19T20:44:30.330 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84724.log 2026-02-19T20:44:30.330 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57308.log.gz 2026-02-19T20:44:30.330 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70070.log 2026-02-19T20:44:30.331 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82068.log: /var/log/ceph/cluster2-client.mirror.84724.log: 87.5%gzip -5 --verbose -- 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84724.log.gz 2026-02-19T20:44:30.331 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster2-client.mirror.76075.log 2026-02-19T20:44:30.331 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.82068.log.gz 2026-02-19T20:44:30.332 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70070.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78307.log 2026-02-19T20:44:30.332 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.70070.log.gz 2026-02-19T20:44:30.332 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76075.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78217.log 2026-02-19T20:44:30.332 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.76075.log.gz 2026-02-19T20:44:30.332 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51010.log 2026-02-19T20:44:30.333 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78307.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61975.log 2026-02-19T20:44:30.333 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78217.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78307.log.gz 2026-02-19T20:44:30.333 INFO:teuthology.orchestra.run.trial080.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.78217.log.gz 2026-02-19T20:44:30.334 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.51010.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72596.log 2026-02-19T20:44:30.334 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.51010.log.gz 2026-02-19T20:44:30.334 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61975.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52616.log 2026-02-19T20:44:30.334 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.61975.log.gz 2026-02-19T20:44:30.334 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72596.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62300.log 2026-02-19T20:44:30.334 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65567.log 2026-02-19T20:44:30.335 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52616.log: /var/log/ceph/cluster1-client.mirror.62300.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62300.log.gz 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.72596.log.gz 2026-02-19T20:44:30.335 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.335 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.52233.log.gz 2026-02-19T20:44:30.335 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74587.log 2026-02-19T20:44:30.335 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65567.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72106.log 2026-02-19T20:44:30.336 INFO:teuthology.orchestra.run.trial080.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.65567.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61758.log 2026-02-19T20:44:30.336 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.336 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74587.log: /var/log/ceph/cluster2-client.mirror.72106.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50414.log 2026-02-19T20:44:30.337 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61758.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74587.log.gz 2026-02-19T20:44:30.337 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.61758.log.gz 2026-02-19T20:44:30.337 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51806.log 2026-02-19T20:44:30.337 INFO:teuthology.orchestra.run.trial080.stderr: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.72106.log.gz 2026-02-19T20:44:30.337 INFO:teuthology.orchestra.run.trial080.stderr: 93.3%/var/log/ceph/cluster2-client.mirror.50414.log: -- replaced with /var/log/ceph/cluster2-client.mirror.52616.log.gz 2026-02-19T20:44:30.338 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-mon.a.log 2026-02-19T20:44:30.338 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51806.log: 87.0% -- replaced with /var/log/ceph/cluster2-client.mirror.50414.log.gz 2026-02-19T20:44:30.338 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71784.log 2026-02-19T20:44:30.338 INFO:teuthology.orchestra.run.trial080.stderr: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.51806.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66220.log 2026-02-19T20:44:30.338 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.338 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71784.log: /var/log/ceph/cluster2-mon.a.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55613.log 2026-02-19T20:44:30.339 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.71784.log.gz 2026-02-19T20:44:30.339 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66220.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57071.log 2026-02-19T20:44:30.339 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74318.log 2026-02-19T20:44:30.340 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55613.log: /var/log/ceph/cluster2-client.mirror.57071.log: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.66220.log.gz 2026-02-19T20:44:30.340 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53207.log.gz 2026-02-19T20:44:30.340 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58056.log.gz 2026-02-19T20:44:30.340 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57387.log 2026-02-19T20:44:30.341 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70627.log 2026-02-19T20:44:30.341 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74318.log: /var/log/ceph/cluster2-client.mirror.57387.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80836.log 2026-02-19T20:44:30.342 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70627.log: 87.5% 90.1% -- replaced with /var/log/ceph/cluster1-client.mirror.70627.log.gz -- replaced with /var/log/ceph/cluster2-client.mirror.74318.log.gz 2026-02-19T20:44:30.342 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.342 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56446.log 2026-02-19T20:44:30.342 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.80836.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61871.log 2026-02-19T20:44:30.343 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.80836.log.gz 2026-02-19T20:44:30.343 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56446.log: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55613.log.gz 2026-02-19T20:44:30.343 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.56446.log.gz 2026-02-19T20:44:30.343 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79140.log 2026-02-19T20:44:30.344 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49146.log 2026-02-19T20:44:30.344 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61871.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67865.log 2026-02-19T20:44:30.344 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79140.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.79140.log.gz 2026-02-19T20:44:30.344 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49146.log: 54.2% -- replaced with /var/log/ceph/cluster2-client.mirror.49146.log.gz 2026-02-19T20:44:30.345 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51614.log 2026-02-19T20:44:30.345 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.61871.log.gz 2026-02-19T20:44:30.345 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77430.log 2026-02-19T20:44:30.345 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67865.log: gzip/var/log/ceph/cluster1-client.mirror.51614.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75372.log 2026-02-19T20:44:30.346 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77430.log: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.77430.log.gz 2026-02-19T20:44:30.346 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54655.log 2026-02-19T20:44:30.346 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75372.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.67865.log.gz 2026-02-19T20:44:30.347 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67252.log 2026-02-19T20:44:30.347 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54655.log: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.75372.log.gz 2026-02-19T20:44:30.348 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63209.log 2026-02-19T20:44:30.348 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67252.log: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.54655.log.gz 2026-02-19T20:44:30.348 INFO:teuthology.orchestra.run.trial080.stderr: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.67252.log.gz 2026-02-19T20:44:30.348 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82638.log 2026-02-19T20:44:30.349 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.51614.log.gz 2026-02-19T20:44:30.349 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.56616.log.gz 2026-02-19T20:44:30.349 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55699.log 2026-02-19T20:44:30.349 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63209.log: /var/log/ceph/cluster2-client.mirror.82638.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79342.log 2026-02-19T20:44:30.349 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.63209.log.gz 2026-02-19T20:44:30.349 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.82638.log.gz 2026-02-19T20:44:30.349 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81205.log 2026-02-19T20:44:30.350 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55699.log: /var/log/ceph/cluster1-client.mirror.79342.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75476.log 2026-02-19T20:44:30.350 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.79342.log.gz 2026-02-19T20:44:30.350 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.81205.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81818.log 2026-02-19T20:44:30.350 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.81205.log.gz 2026-02-19T20:44:30.350 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76426.log 2026-02-19T20:44:30.350 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75476.log: /var/log/ceph/cluster2-client.mirror.81818.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78105.log 2026-02-19T20:44:30.351 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76426.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.75476.log.gz 2026-02-19T20:44:30.351 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.81818.log.gz 2026-02-19T20:44:30.351 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56226.log 2026-02-19T20:44:30.351 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81162.log 2026-02-19T20:44:30.351 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78105.log: /var/log/ceph/cluster2-client.mirror.56226.log: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.76426.log.gz 2026-02-19T20:44:30.352 INFO:teuthology.orchestra.run.trial080.stderr: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.78105.log.gz 2026-02-19T20:44:30.352 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- 87.8% /var/log/ceph/cluster2-client.mirror.79744.log 2026-02-19T20:44:30.352 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.56226.log.gz 2026-02-19T20:44:30.352 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81162.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53300.log 2026-02-19T20:44:30.352 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.81162.log.gz 2026-02-19T20:44:30.352 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54914.log 2026-02-19T20:44:30.353 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79744.log: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55699.log.gz 2026-02-19T20:44:30.353 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53300.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49185.log 2026-02-19T20:44:30.353 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.79744.log.gz 2026-02-19T20:44:30.353 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54914.log: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.53300.log.gz 2026-02-19T20:44:30.353 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81560.log 2026-02-19T20:44:30.353 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster1-client.mirror.54914.log.gz 2026-02-19T20:44:30.354 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49185.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68559.log 2026-02-19T20:44:30.354 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81560.log: 88.6%gzip -- replaced with /var/log/ceph/cluster2-client.mirror.49185.log.gz 2026-02-19T20:44:30.354 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster1-client.admin.52339.log 2026-02-19T20:44:30.355 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68559.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55247.log 2026-02-19T20:44:30.355 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68559.log.gz 2026-02-19T20:44:30.355 INFO:teuthology.orchestra.run.trial080.stderr: 90.7% -- replaced with /var/log/ceph/cluster2-client.mirror.81560.log.gz 2026-02-19T20:44:30.355 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.52339.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72748.log 2026-02-19T20:44:30.355 INFO:teuthology.orchestra.run.trial080.stderr: 87.8%/var/log/ceph/cluster2-client.mirror.55247.log: -- replaced with /var/log/ceph/cluster1-client.admin.52339.log.gzgzip 2026-02-19T20:44:30.355 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63408.log 2026-02-19T20:44:30.355 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72748.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57029.log 2026-02-19T20:44:30.355 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.72748.log.gz 2026-02-19T20:44:30.355 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85476.log 2026-02-19T20:44:30.356 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63408.log: /var/log/ceph/cluster2-client.mirror.57029.log: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.63408.log.gz 2026-02-19T20:44:30.356 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.57029.log.gz 2026-02-19T20:44:30.356 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster1-client.mirror.65906.log 2026-02-19T20:44:30.356 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85476.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85476.log.gz 2026-02-19T20:44:30.357 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54681.log 2026-02-19T20:44:30.357 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.65906.log: -- /var/log/ceph/cluster1-client.mirror.53462.log 2026-02-19T20:44:30.357 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54681.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.65906.log.gz 2026-02-19T20:44:30.357 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79573.log 2026-02-19T20:44:30.357 INFO:teuthology.orchestra.run.trial080.stderr: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.54681.log.gz 2026-02-19T20:44:30.358 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53462.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61782.log 2026-02-19T20:44:30.358 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55247.log.gz/var/log/ceph/cluster2-client.mirror.79573.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77700.log 2026-02-19T20:44:30.358 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster1-client.mirror.53462.log.gz 2026-02-19T20:44:30.358 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.358 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61782.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.61782.log.gz 2026-02-19T20:44:30.359 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.50309.log 2026-02-19T20:44:30.359 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77700.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68937.log 2026-02-19T20:44:30.359 INFO:teuthology.orchestra.run.trial080.stderr: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.77700.log.gz 2026-02-19T20:44:30.359 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.50309.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66103.log 2026-02-19T20:44:30.360 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.admin.50309.log.gz 2026-02-19T20:44:30.360 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68937.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84778.log 2026-02-19T20:44:30.360 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.68937.log.gz 2026-02-19T20:44:30.360 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83798.log 2026-02-19T20:44:30.360 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66103.log: 87.1% -- replaced with /var/log/ceph/cluster2-client.mirror.66103.log.gz 2026-02-19T20:44:30.360 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84778.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84778.log.gz 2026-02-19T20:44:30.361 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55808.log 2026-02-19T20:44:30.361 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83798.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64680.log 2026-02-19T20:44:30.361 INFO:teuthology.orchestra.run.trial080.stderr: 86.8%/var/log/ceph/cluster2-client.mirror.55808.log: -- replaced with /var/log/ceph/cluster2-client.mirror.83798.log.gz 2026-02-19T20:44:30.361 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67567.log 2026-02-19T20:44:30.362 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose/var/log/ceph/cluster2-client.mirror.64680.log: -- /var/log/ceph/cluster2-client.mirror.75021.log 2026-02-19T20:44:30.362 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.64680.log.gz 2026-02-19T20:44:30.362 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67567.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.67567.log.gz 2026-02-19T20:44:30.364 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55808.log.gz 2026-02-19T20:44:30.364 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69319.log 2026-02-19T20:44:30.365 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75021.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81499.log 2026-02-19T20:44:30.365 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69319.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.75021.log.gz 2026-02-19T20:44:30.365 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56146.log 2026-02-19T20:44:30.366 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81499.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81499.log.gz 2026-02-19T20:44:30.366 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74918.log 2026-02-19T20:44:30.366 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.69319.log.gz 2026-02-19T20:44:30.366 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56146.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59457.log 2026-02-19T20:44:30.366 INFO:teuthology.orchestra.run.trial080.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56146.log.gz 2026-02-19T20:44:30.366 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56346.log 2026-02-19T20:44:30.367 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74918.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.74918.log.gz 2026-02-19T20:44:30.373 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59457.log: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.59457.log.gz 2026-02-19T20:44:30.373 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.50185.log 2026-02-19T20:44:30.374 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56346.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77677.log 2026-02-19T20:44:30.374 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60160.log 2026-02-19T20:44:30.374 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.0.50185.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56346.log.gz 2026-02-19T20:44:30.375 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77677.log: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.77677.log.gz 2026-02-19T20:44:30.375 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53340.log 2026-02-19T20:44:30.375 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60160.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82757.log 2026-02-19T20:44:30.376 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53340.log: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.53340.log.gz 2026-02-19T20:44:30.376 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46657.log 2026-02-19T20:44:30.376 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82757.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.82757.log.gz 2026-02-19T20:44:30.377 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47854.log 2026-02-19T20:44:30.377 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.0.50185.log.gz 2026-02-19T20:44:30.377 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46657.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46657.log.gz 2026-02-19T20:44:30.377 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49061.log 2026-02-19T20:44:30.377 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81478.log 2026-02-19T20:44:30.378 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.47854.log: /var/log/ceph/cluster2-client.mirror.49061.log: 75.0% -- replaced with /var/log/ceph/cluster2-client.mirror.47854.log.gz 2026-02-19T20:44:30.378 INFO:teuthology.orchestra.run.trial080.stderr: 2.3% -- replaced with /var/log/ceph/cluster2-client.mirror.49061.log.gz 2026-02-19T20:44:30.378 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72724.log 2026-02-19T20:44:30.378 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62178.log 2026-02-19T20:44:30.379 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81478.log: /var/log/ceph/cluster1-client.mirror.72724.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81478.log.gz 2026-02-19T20:44:30.379 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.72724.log.gz 2026-02-19T20:44:30.379 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73466.log 2026-02-19T20:44:30.379 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66344.log 2026-02-19T20:44:30.380 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62178.log: /var/log/ceph/cluster1-client.mirror.73466.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.62178.log.gz 2026-02-19T20:44:30.380 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73056.log 2026-02-19T20:44:30.381 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66344.log: 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.73466.log.gz 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.66344.log.gz 2026-02-19T20:44:30.381 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.381 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52593.log 2026-02-19T20:44:30.381 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58821.log 2026-02-19T20:44:30.381 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73056.log: /var/log/ceph/cluster1-client.mirror.52593.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.73056.log.gz 2026-02-19T20:44:30.382 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60052.log 2026-02-19T20:44:30.382 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58821.log: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58821.log.gz 2026-02-19T20:44:30.383 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70159.log 2026-02-19T20:44:30.383 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60052.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60052.log.gz 2026-02-19T20:44:30.383 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62092.log 2026-02-19T20:44:30.385 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70159.log: 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.70159.log.gz 2026-02-19T20:44:30.385 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.52593.log.gz 2026-02-19T20:44:30.385 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48447.log 2026-02-19T20:44:30.385 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81335.log 2026-02-19T20:44:30.386 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62092.log: /var/log/ceph/cluster1-client.admin.48447.log: 80.1% -- replaced with /var/log/ceph/cluster1-client.admin.48447.log.gz 2026-02-19T20:44:30.386 INFO:teuthology.orchestra.run.trial080.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.62092.log.gz 2026-02-19T20:44:30.387 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60958.log 2026-02-19T20:44:30.387 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59867.log 2026-02-19T20:44:30.387 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81335.log: /var/log/ceph/cluster2-client.mirror.60958.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81335.log.gz 2026-02-19T20:44:30.388 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.60958.log.gz 2026-02-19T20:44:30.388 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74401.log 2026-02-19T20:44:30.388 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63159.log 2026-02-19T20:44:30.388 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59867.log: /var/log/ceph/cluster2-client.mirror.74401.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.59867.log.gz 2026-02-19T20:44:30.389 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74401.log.gz 2026-02-19T20:44:30.389 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53510.log 2026-02-19T20:44:30.389 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63159.log: 89.0% -- replaced with /var/log/ceph/cluster2-client.mirror.63159.log.gz 2026-02-19T20:44:30.390 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73446.log 2026-02-19T20:44:30.390 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81598.log 2026-02-19T20:44:30.390 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53510.log: 84.3% -- replaced with /var/log/ceph/cluster2-client.mirror.53510.log.gz 2026-02-19T20:44:30.390 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73446.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84050.log 2026-02-19T20:44:30.391 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81598.log: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81598.log.gz 2026-02-19T20:44:30.391 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.73446.log.gz 2026-02-19T20:44:30.392 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53969.log 2026-02-19T20:44:30.392 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69845.log 2026-02-19T20:44:30.392 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84050.log: /var/log/ceph/cluster1-client.mirror.53969.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84050.log.gz 2026-02-19T20:44:30.392 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster1-client.mirror.53969.log.gz 2026-02-19T20:44:30.392 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70579.log 2026-02-19T20:44:30.393 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58373.log 2026-02-19T20:44:30.393 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69845.log: /var/log/ceph/cluster1-client.mirror.70579.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.69845.log.gz 2026-02-19T20:44:30.393 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.70579.log.gz 2026-02-19T20:44:30.393 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80472.log 2026-02-19T20:44:30.394 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76639.log 2026-02-19T20:44:30.394 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58373.log: /var/log/ceph/cluster2-client.mirror.80472.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.80472.log.gz 2026-02-19T20:44:30.401 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59687.log 2026-02-19T20:44:30.402 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76639.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.76639.log.gz 2026-02-19T20:44:30.406 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.60980.log.gz 2026-02-19T20:44:30.418 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57918.log 2026-02-19T20:44:30.419 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59687.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65978.log 2026-02-19T20:44:30.419 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.59687.log.gz 2026-02-19T20:44:30.419 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57918.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72214.log 2026-02-19T20:44:30.419 INFO:teuthology.orchestra.run.trial080.stderr: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.57918.log.gz 2026-02-19T20:44:30.420 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65978.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71853.log 2026-02-19T20:44:30.420 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65978.log.gz 2026-02-19T20:44:30.420 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72214.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48123.log 2026-02-19T20:44:30.420 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.72214.log.gz 2026-02-19T20:44:30.421 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71853.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78240.log 2026-02-19T20:44:30.421 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48123.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.61069.log.gz 2026-02-19T20:44:30.422 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.71853.log.gz 2026-02-19T20:44:30.422 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55204.log 2026-02-19T20:44:30.422 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78240.log: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.78240.log.gz 2026-02-19T20:44:30.422 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47392.log 2026-02-19T20:44:30.423 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55204.log: 95.0% -- replaced with /var/log/ceph/cluster1-client.mirror.48123.log.gz 2026-02-19T20:44:30.425 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55204.log.gz 2026-02-19T20:44:30.435 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.60262.log 2026-02-19T20:44:30.436 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.47392.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61097.log 2026-02-19T20:44:30.436 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47392.log.gz 2026-02-19T20:44:30.436 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.60262.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56984.log 2026-02-19T20:44:30.437 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58611.log.gz/var/log/ceph/cluster2-client.mirror.61097.log: 2026-02-19T20:44:30.437 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69381.log 2026-02-19T20:44:30.437 INFO:teuthology.orchestra.run.trial080.stderr: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.61097.log.gz 2026-02-19T20:44:30.437 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56984.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62276.log 2026-02-19T20:44:30.437 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69381.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73080.log 2026-02-19T20:44:30.438 INFO:teuthology.orchestra.run.trial080.stderr: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.69381.log.gz 2026-02-19T20:44:30.438 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62276.log: 95.2% -- replaced with /var/log/ceph/cluster1-client.mirror.60262.log.gz 2026-02-19T20:44:30.438 INFO:teuthology.orchestra.run.trial080.stderr: 87.4%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60138.log 2026-02-19T20:44:30.438 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.62276.log.gz 2026-02-19T20:44:30.440 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73080.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.73080.log.gz 2026-02-19T20:44:30.440 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster2-client.mirror.73488.log 2026-02-19T20:44:30.440 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60138.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82474.log 2026-02-19T20:44:30.440 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60138.log.gz 2026-02-19T20:44:30.440 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73488.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68812.log 2026-02-19T20:44:30.440 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.73488.log.gz 2026-02-19T20:44:30.440 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77159.log 2026-02-19T20:44:30.440 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82474.log: /var/log/ceph/cluster1-client.mirror.68812.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51094.log 2026-02-19T20:44:30.441 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.82474.log.gz 2026-02-19T20:44:30.441 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.77159.log: 87.7% -- replaced with /var/log/ceph/cluster1-client.mirror.68812.log.gz 2026-02-19T20:44:30.441 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77227.log 2026-02-19T20:44:30.441 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.53781.log 2026-02-19T20:44:30.441 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.51094.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.77159.log.gz 2026-02-19T20:44:30.442 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77227.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77227.log.gz 2026-02-19T20:44:30.442 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60626.log 2026-02-19T20:44:30.442 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.53781.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65525.log 2026-02-19T20:44:30.443 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60626.log: 87.8% -- replaced with /var/log/ceph/cluster1-client.admin.53781.log.gz 2026-02-19T20:44:30.443 INFO:teuthology.orchestra.run.trial080.stderr: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60626.log.gz 2026-02-19T20:44:30.443 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67907.log 2026-02-19T20:44:30.443 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65525.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.52754.log 2026-02-19T20:44:30.444 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67907.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.67907.log.gz 2026-02-19T20:44:30.444 INFO:teuthology.orchestra.run.trial080.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.65525.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84006.log 2026-02-19T20:44:30.444 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.445 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.52754.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75806.log 2026-02-19T20:44:30.445 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.84006.log: 87.8% -- replaced with /var/log/ceph/cluster1-client.admin.52754.log.gz 2026-02-19T20:44:30.445 INFO:teuthology.orchestra.run.trial080.stderr: 86.7% -- replaced with /var/log/ceph/cluster1-client.mirror.84006.log.gz 2026-02-19T20:44:30.445 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60663.log 2026-02-19T20:44:30.446 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.51094.log.gz 2026-02-19T20:44:30.446 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75806.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66953.log 2026-02-19T20:44:30.446 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60663.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.60663.log.gz 2026-02-19T20:44:30.447 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.75806.log.gz 2026-02-19T20:44:30.447 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58034.log 2026-02-19T20:44:30.447 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66953.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74670.log 2026-02-19T20:44:30.447 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58034.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.56043.log 2026-02-19T20:44:30.448 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58034.log.gz 2026-02-19T20:44:30.448 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74670.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.66953.log.gz 2026-02-19T20:44:30.448 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66368.log 2026-02-19T20:44:30.448 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.74670.log.gz 2026-02-19T20:44:30.449 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.56043.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77520.log 2026-02-19T20:44:30.450 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66368.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56487.log 2026-02-19T20:44:30.450 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.66368.log.gz 2026-02-19T20:44:30.450 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77520.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81519.log 2026-02-19T20:44:30.450 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.77520.log.gz 2026-02-19T20:44:30.451 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56487.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53163.log 2026-02-19T20:44:30.451 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81519.log: 90.4% -- replaced with /var/log/ceph/cluster2-client.mirror.56487.log.gz 2026-02-19T20:44:30.451 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81519.log.gz 2026-02-19T20:44:30.451 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65701.log 2026-02-19T20:44:30.451 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53163.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74194.log 2026-02-19T20:44:30.451 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.56043.log.gz 2026-02-19T20:44:30.452 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65701.log: 87.6%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71495.log 2026-02-19T20:44:30.452 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.65701.log.gz 2026-02-19T20:44:30.452 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74194.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79095.log 2026-02-19T20:44:30.453 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71495.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74194.log.gz 2026-02-19T20:44:30.453 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.71495.log.gz 2026-02-19T20:44:30.453 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80451.log 2026-02-19T20:44:30.454 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79095.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61951.log 2026-02-19T20:44:30.454 INFO:teuthology.orchestra.run.trial080.stderr: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.79095.log.gz 2026-02-19T20:44:30.455 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80451.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63936.log 2026-02-19T20:44:30.455 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.53163.log.gz 2026-02-19T20:44:30.455 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61951.log: 86.7% -- replaced with /var/log/ceph/cluster2-client.mirror.80451.log.gz 2026-02-19T20:44:30.455 INFO:teuthology.orchestra.run.trial080.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59371.log 2026-02-19T20:44:30.455 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.61951.log.gz 2026-02-19T20:44:30.456 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63936.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66417.log 87.3% 2026-02-19T20:44:30.456 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.63936.log.gz 2026-02-19T20:44:30.456 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59371.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64507.log 2026-02-19T20:44:30.456 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66417.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59371.log.gz 2026-02-19T20:44:30.456 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.66417.log.gz 2026-02-19T20:44:30.456 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55134.log 2026-02-19T20:44:30.457 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64507.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83120.log 2026-02-19T20:44:30.457 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.64507.log.gz 2026-02-19T20:44:30.457 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55134.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79638.log 2026-02-19T20:44:30.457 INFO:teuthology.orchestra.run.trial080.stderr: 89.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55134.log.gz 2026-02-19T20:44:30.457 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.2.59457.log 2026-02-19T20:44:30.458 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.83120.log: /var/log/ceph/cluster2-client.mirror.79638.log: 85.5% -- replaced with /var/log/ceph/cluster1-client.mirror.83120.log.gz 2026-02-19T20:44:30.458 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57407.log 2026-02-19T20:44:30.458 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.79638.log.gz 2026-02-19T20:44:30.459 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.2.59457.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70651.log 2026-02-19T20:44:30.459 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57407.log: 79.3%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53847.log 2026-02-19T20:44:30.459 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.57407.log.gz 2026-02-19T20:44:30.459 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70651.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.85009.log 2026-02-19T20:44:30.460 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.70651.log.gz 2026-02-19T20:44:30.460 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53847.log: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.53847.log.gz 2026-02-19T20:44:30.466 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67302.log 2026-02-19T20:44:30.466 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82911.log 2026-02-19T20:44:30.466 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.85009.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.85009.log.gz 2026-02-19T20:44:30.466 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67302.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74752.log 2026-02-19T20:44:30.467 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.67302.log.gz 2026-02-19T20:44:30.467 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82911.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79005.log 2026-02-19T20:44:30.467 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.82911.log.gz/var/log/ceph/cluster2-client.mirror.74752.log: 2026-02-19T20:44:30.468 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52057.log 2026-02-19T20:44:30.468 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79005.log: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.74752.log.gz 2026-02-19T20:44:30.468 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.79005.log.gz 2026-02-19T20:44:30.469 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84556.log 2026-02-19T20:44:30.469 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52057.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55550.log 2026-02-19T20:44:30.469 INFO:teuthology.orchestra.run.trial080.stderr: 88.0% -- replaced with /var/log/ceph/cluster1-client.mirror.52057.log.gz 2026-02-19T20:44:30.470 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84556.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.56897.log.gz 2026-02-19T20:44:30.470 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51985.log 2026-02-19T20:44:30.470 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55550.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.84556.log.gz 2026-02-19T20:44:30.470 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57738.log.gz 2026-02-19T20:44:30.471 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- 92.7% -- replaced with /var/log/ceph/cluster1-client.mirror.2.59457.log.gz 2026-02-19T20:44:30.471 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster2-client.mirror.55060.log 2026-02-19T20:44:30.472 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51985.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74028.log 2026-02-19T20:44:30.472 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.51985.log.gz 2026-02-19T20:44:30.472 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55060.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78352.log 2026-02-19T20:44:30.474 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74028.log: 89.5% -- replaced with /var/log/ceph/cluster2-client.mirror.55060.log.gz 2026-02-19T20:44:30.474 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57365.log 2026-02-19T20:44:30.474 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.74028.log.gz 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78352.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57977.log 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78352.log.gz 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57365.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62228.log 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57365.log.gz 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57977.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55550.log.gz 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72616.log 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62228.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68369.log 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62228.log.gz 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72616.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56266.log 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68369.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58394.log 2026-02-19T20:44:30.475 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.68369.log.gz/var/log/ceph/cluster2-client.mirror.56266.log: 2026-02-19T20:44:30.476 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77250.log 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% 87.7% -- replaced with /var/log/ceph/cluster2-client.mirror.56266.log.gz 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.72616.log.gz 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr:gzip/var/log/ceph/cluster2-client.mirror.58394.log: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69009.log 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.58394.log.gz 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose --/var/log/ceph/cluster1-client.mirror.77250.log: /var/log/ceph/cluster1-client.mirror.70848.log 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr: 86.3%/var/log/ceph/cluster1-client.mirror.69009.log: -- replaced with /var/log/ceph/cluster1-client.mirror.77250.log.gz 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78037.log 87.0% -- replaced with /var/log/ceph/cluster1-client.mirror.69009.log.gz 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70848.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71591.log 2026-02-19T20:44:30.477 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.70848.log.gz 2026-02-19T20:44:30.478 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78037.log: gzip 86.5% -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76013.log 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.78037.log.gz 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71591.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57424.log 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr: 87.6%/var/log/ceph/cluster2-client.mirror.76013.log: -- replaced with /var/log/ceph/cluster1-client.mirror.71591.log.gz 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83014.log 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76013.log.gz 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57424.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73104.log 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57424.log.gz 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83014.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55828.log 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.83014.log.gz 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73104.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84798.log 2026-02-19T20:44:30.479 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.73104.log.gz 2026-02-19T20:44:30.480 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55828.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74050.log 2026-02-19T20:44:30.480 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84798.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58314.log 2026-02-19T20:44:30.480 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74050.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68666.log 2026-02-19T20:44:30.480 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.84798.log.gz 2026-02-19T20:44:30.480 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74050.log.gz 2026-02-19T20:44:30.480 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58314.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58314.log.gz 2026-02-19T20:44:30.480 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46809.log 2026-02-19T20:44:30.481 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78442.log 2026-02-19T20:44:30.481 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68666.log: /var/log/ceph/cluster2-client.admin.46809.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59199.log 2026-02-19T20:44:30.481 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46809.log.gz 2026-02-19T20:44:30.481 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68666.log.gz 2026-02-19T20:44:30.481 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78442.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.78442.log.gz 2026-02-19T20:44:30.482 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72961.log 2026-02-19T20:44:30.482 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59199.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57091.log 2026-02-19T20:44:30.482 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55828.log.gz 2026-02-19T20:44:30.482 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose --/var/log/ceph/cluster1-client.mirror.72961.log: /var/log/ceph/cluster2-client.mirror.58858.log 2026-02-19T20:44:30.482 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59199.log.gz 2026-02-19T20:44:30.483 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57091.log: 94.2%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74959.log 2026-02-19T20:44:30.483 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.72961.log.gz 2026-02-19T20:44:30.483 INFO:teuthology.orchestra.run.trial080.stderr: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.57091.log.gz 2026-02-19T20:44:30.483 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58858.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67495.log 2026-02-19T20:44:30.483 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.58858.log.gz 2026-02-19T20:44:30.483 INFO:teuthology.orchestra.run.trial080.stderr:gzip/var/log/ceph/cluster2-client.mirror.74959.log: -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78150.log 2026-02-19T20:44:30.484 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67495.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74959.log.gz 2026-02-19T20:44:30.484 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59100.log 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.67495.log.gz 2026-02-19T20:44:30.484 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.485 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78150.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80747.log 2026-02-19T20:44:30.485 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78150.log.gz 2026-02-19T20:44:30.485 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59100.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84959.log 2026-02-19T20:44:30.485 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.80747.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59100.log.gz 2026-02-19T20:44:30.486 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.1.80151.log 2026-02-19T20:44:30.486 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.80747.log.gz 2026-02-19T20:44:30.486 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.84959.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84959.log.gz 2026-02-19T20:44:30.486 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65882.log 2026-02-19T20:44:30.486 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.1.80151.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52966.log 2026-02-19T20:44:30.487 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65882.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77812.log 2026-02-19T20:44:30.487 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.65882.log.gz 2026-02-19T20:44:30.487 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52966.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78667.log 2026-02-19T20:44:30.487 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster1-client.mirror.52966.log.gz 2026-02-19T20:44:30.487 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54467.log 2026-02-19T20:44:30.487 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77812.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.77812.log.gz 2026-02-19T20:44:30.487 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52479.log 2026-02-19T20:44:30.488 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78667.log: /var/log/ceph/cluster2-client.mirror.54467.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78667.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82699.log 2026-02-19T20:44:30.488 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.488 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52479.log: 88.4% -- replaced with /var/log/ceph/cluster1-client.mirror.52479.log.gz 2026-02-19T20:44:30.489 INFO:teuthology.orchestra.run.trial080.stderr: 93.6% -- replaced with /var/log/ceph/cluster1-client.mirror.1.80151.log.gz 2026-02-19T20:44:30.490 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54978.log 2026-02-19T20:44:30.490 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82699.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73353.log 2026-02-19T20:44:30.491 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54978.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77542.log 2026-02-19T20:44:30.491 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% 90.7% -- replaced with /var/log/ceph/cluster2-client.mirror.82699.log.gz 2026-02-19T20:44:30.491 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.54467.log.gz 2026-02-19T20:44:30.491 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73353.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77992.log 2026-02-19T20:44:30.491 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.73353.log.gz 2026-02-19T20:44:30.491 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77542.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61156.log 2026-02-19T20:44:30.492 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77542.log.gz 2026-02-19T20:44:30.492 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77992.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49044.log 2026-02-19T20:44:30.492 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77992.log.gz 2026-02-19T20:44:30.492 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76509.log 2026-02-19T20:44:30.493 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61156.log: /var/log/ceph/cluster2-client.mirror.49044.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68511.log 2026-02-19T20:44:30.493 INFO:teuthology.orchestra.run.trial080.stderr: 2.3% -- replaced with /var/log/ceph/cluster2-client.mirror.49044.log.gz 2026-02-19T20:44:30.493 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76509.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78622.log 2026-02-19T20:44:30.493 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76509.log.gz 2026-02-19T20:44:30.494 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68511.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68511.log.gz 2026-02-19T20:44:30.494 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.54978.log.gz 2026-02-19T20:44:30.494 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54331.log 2026-02-19T20:44:30.494 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78622.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70181.log 2026-02-19T20:44:30.494 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78622.log.gz 2026-02-19T20:44:30.495 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54331.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66658.log 2026-02-19T20:44:30.495 INFO:teuthology.orchestra.run.trial080.stderr: 88.8% -- replaced with /var/log/ceph/cluster1-client.mirror.54331.log.gz 2026-02-19T20:44:30.495 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70181.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66262.log 2026-02-19T20:44:30.495 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66658.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58156.log 2026-02-19T20:44:30.495 INFO:teuthology.orchestra.run.trial080.stderr: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.70181.log.gz 2026-02-19T20:44:30.496 INFO:teuthology.orchestra.run.trial080.stderr: 87.8% -- replaced with /var/log/ceph/cluster1-client.mirror.66658.log.gz 2026-02-19T20:44:30.496 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66262.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.66262.log.gz 2026-02-19T20:44:30.502 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.48941.log 2026-02-19T20:44:30.502 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58156.log: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.58156.log.gz 2026-02-19T20:44:30.502 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73630.log 2026-02-19T20:44:30.502 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.48941.log: gzip -5 --verbose -- 80.4% -- replaced with /var/log/ceph/cluster2-client.admin.48941.log.gz 2026-02-19T20:44:30.502 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster2-client.admin.47196.log 2026-02-19T20:44:30.503 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73630.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78757.log 2026-02-19T20:44:30.503 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.73630.log.gz 2026-02-19T20:44:30.503 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47196.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71270.log 2026-02-19T20:44:30.503 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47196.log.gz 2026-02-19T20:44:30.503 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74112.log 2026-02-19T20:44:30.504 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78757.log: /var/log/ceph/cluster1-client.mirror.71270.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80151.log 2026-02-19T20:44:30.504 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78757.log.gz 2026-02-19T20:44:30.504 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.71270.log.gz 2026-02-19T20:44:30.504 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74112.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.1.60262.log 2026-02-19T20:44:30.504 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74112.log.gz 2026-02-19T20:44:30.504 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73176.log 2026-02-19T20:44:30.505 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80151.log: /var/log/ceph/cluster2-client.mirror.1.60262.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83484.log 2026-02-19T20:44:30.505 INFO:teuthology.orchestra.run.trial080.stderr: 84.8% -- replaced with /var/log/ceph/cluster2-client.mirror.80151.log.gz 2026-02-19T20:44:30.505 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73176.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.73176.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69182.log 2026-02-19T20:44:30.506 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.506 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.83484.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53873.log 2026-02-19T20:44:30.506 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69182.log: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.83484.log.gz 2026-02-19T20:44:30.506 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.69182.log.gz 2026-02-19T20:44:30.516 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71246.log 2026-02-19T20:44:30.516 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53873.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80513.log 2026-02-19T20:44:30.516 INFO:teuthology.orchestra.run.trial080.stderr: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53873.log.gz 2026-02-19T20:44:30.517 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71246.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54177.log 2026-02-19T20:44:30.517 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80513.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.71246.log.gz 2026-02-19T20:44:30.517 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.80513.log.gz 2026-02-19T20:44:30.517 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71166.log 2026-02-19T20:44:30.518 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.54177.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79479.log 2026-02-19T20:44:30.518 INFO:teuthology.orchestra.run.trial080.stderr: 88.1% -- replaced with /var/log/ceph/cluster1-client.admin.54177.log.gz 2026-02-19T20:44:30.518 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71166.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64326.log 2026-02-19T20:44:30.519 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79479.log: 41.6% -- replaced with /var/log/ceph/cluster1-client.mirror.79479.log.gz 2026-02-19T20:44:30.519 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose 92.1% -- /var/log/ceph/cluster1-client.mirror.69544.log 2026-02-19T20:44:30.519 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.71166.log.gz 2026-02-19T20:44:30.520 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64326.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79785.log 2026-02-19T20:44:30.521 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69544.log: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.64326.log.gz 2026-02-19T20:44:30.521 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.69544.log.gz 2026-02-19T20:44:30.521 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51458.log 2026-02-19T20:44:30.521 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67374.log 2026-02-19T20:44:30.522 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79785.log: /var/log/ceph/cluster1-client.mirror.51458.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.79785.log.gz 2026-02-19T20:44:30.522 INFO:teuthology.orchestra.run.trial080.stderr: 87.7% -- replaced with /var/log/ceph/cluster1-client.mirror.51458.log.gz 2026-02-19T20:44:30.522 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74091.log 2026-02-19T20:44:30.523 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.47901.log 2026-02-19T20:44:30.523 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67374.log: 87.5%/var/log/ceph/cluster2-client.mirror.74091.log: -- replaced with /var/log/ceph/cluster1-client.mirror.67374.log.gz 2026-02-19T20:44:30.524 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.74091.log.gz 2026-02-19T20:44:30.524 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51594.log 2026-02-19T20:44:30.524 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.47901.log: 76.8% -- replaced with /var/log/ceph/cluster2-client.mirror.47901.log.gz 2026-02-19T20:44:30.524 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84984.log 2026-02-19T20:44:30.524 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76659.log 2026-02-19T20:44:30.525 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51594.log: /var/log/ceph/cluster2-client.mirror.84984.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.mirror.84984.log.gz 2026-02-19T20:44:30.525 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60373.log 2026-02-19T20:44:30.526 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76659.log: 90.2% -- replaced with /var/log/ceph/cluster2-client.mirror.76659.log.gz 2026-02-19T20:44:30.527 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75662.log 2026-02-19T20:44:30.528 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60373.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.51594.log.gz 2026-02-19T20:44:30.528 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55914.log 2026-02-19T20:44:30.529 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75662.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.75662.log.gz 2026-02-19T20:44:30.529 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63554.log 2026-02-19T20:44:30.532 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55914.log: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55914.log.gz 2026-02-19T20:44:30.533 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67350.log 2026-02-19T20:44:30.533 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63554.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.63554.log.gz 2026-02-19T20:44:30.534 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81314.log 2026-02-19T20:44:30.534 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67350.log: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.67350.log.gz 2026-02-19T20:44:30.535 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67278.log 2026-02-19T20:44:30.535 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81314.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.81314.log.gz 2026-02-19T20:44:30.535 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81437.log 2026-02-19T20:44:30.536 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67278.log: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.67278.log.gz 2026-02-19T20:44:30.537 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71146.log 2026-02-19T20:44:30.537 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81437.log: 86.7% -- replaced with /var/log/ceph/cluster2-client.mirror.81437.log.gz 2026-02-19T20:44:30.538 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67133.log 2026-02-19T20:44:30.539 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71146.log: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.71146.log.gz 2026-02-19T20:44:30.540 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69431.log 2026-02-19T20:44:30.540 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67133.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.67133.log.gz 2026-02-19T20:44:30.541 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52455.log 2026-02-19T20:44:30.541 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69431.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.69431.log.gz 2026-02-19T20:44:30.542 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.82254.log 2026-02-19T20:44:30.542 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52455.log: 88.1% -- replaced with /var/log/ceph/cluster1-client.mirror.52455.log.gz 2026-02-19T20:44:30.542 INFO:teuthology.orchestra.run.trial080.stderr: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.1.60262.log.gz 2026-02-19T20:44:30.543 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.51317.log 2026-02-19T20:44:30.543 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64305.log 2026-02-19T20:44:30.544 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.0.82254.log: /var/log/ceph/cluster1-client.admin.51317.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.admin.51317.log.gz 2026-02-19T20:44:30.544 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54307.log 2026-02-19T20:44:30.545 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64305.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.64305.log.gz 2026-02-19T20:44:30.545 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72262.log 2026-02-19T20:44:30.546 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54307.log: 92.9% -- replaced with /var/log/ceph/cluster1-client.mirror.0.82254.log.gz 2026-02-19T20:44:30.546 INFO:teuthology.orchestra.run.trial080.stderr: 89.1% -- replaced with /var/log/ceph/cluster1-client.mirror.54307.log.gz 2026-02-19T20:44:30.546 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70872.log 2026-02-19T20:44:30.546 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72262.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74484.log 2026-02-19T20:44:30.547 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.72262.log.gz 2026-02-19T20:44:30.547 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70872.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51384.log 2026-02-19T20:44:30.548 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74484.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.70872.log.gz 2026-02-19T20:44:30.548 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74484.log.gz 2026-02-19T20:44:30.548 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55396.log 2026-02-19T20:44:30.548 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51384.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60351.log 2026-02-19T20:44:30.549 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55396.log: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.51384.log.gz 2026-02-19T20:44:30.549 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73329.log 2026-02-19T20:44:30.550 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60351.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60351.log.gz 2026-02-19T20:44:30.550 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71686.log 2026-02-19T20:44:30.550 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73329.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.73329.log.gz 2026-02-19T20:44:30.551 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54614.log 2026-02-19T20:44:30.551 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71686.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.71686.log.gz 2026-02-19T20:44:30.552 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76468.log 2026-02-19T20:44:30.552 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54614.log: 89.6% -- replaced with /var/log/ceph/cluster2-client.mirror.54614.log.gz 2026-02-19T20:44:30.553 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55396.log.gz 2026-02-19T20:44:30.553 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75310.log 2026-02-19T20:44:30.553 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76468.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60101.log 2026-02-19T20:44:30.554 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.76468.log.gz 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59806.log.gz 2026-02-19T20:44:30.554 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.554 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75310.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84153.log 2026-02-19T20:44:30.554 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60101.log: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.60101.log.gz 2026-02-19T20:44:30.554 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52169.log 2026-02-19T20:44:30.555 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.75310.log.gz 2026-02-19T20:44:30.555 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster2-client.mirror.76199.log 2026-02-19T20:44:30.555 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84153.log: /var/log/ceph/cluster2-client.mirror.52169.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56086.log 2026-02-19T20:44:30.556 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76199.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84153.log.gz 2026-02-19T20:44:30.556 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.76199.log.gz 2026-02-19T20:44:30.556 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64008.log 2026-02-19T20:44:30.556 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83291.log 2026-02-19T20:44:30.556 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56086.log: /var/log/ceph/cluster1-client.mirror.64008.log: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.64008.log.gz 2026-02-19T20:44:30.557 INFO:teuthology.orchestra.run.trial080.stderr: 88.1% -- replaced with /var/log/ceph/cluster2-client.mirror.56086.log.gz 2026-02-19T20:44:30.557 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61047.log 2026-02-19T20:44:30.557 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83291.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79162.log 2026-02-19T20:44:30.557 INFO:teuthology.orchestra.run.trial080.stderr: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.83291.log.gz 2026-02-19T20:44:30.558 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61047.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80431.log 2026-02-19T20:44:30.558 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79162.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.79162.log.gz 2026-02-19T20:44:30.558 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61047.log.gz 2026-02-19T20:44:30.558 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.52169.log.gz 2026-02-19T20:44:30.559 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62847.log 2026-02-19T20:44:30.559 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80431.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83614.log 2026-02-19T20:44:30.559 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.83920.log.gz 2026-02-19T20:44:30.559 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.80431.log.gz/var/log/ceph/cluster1-client.mirror.62847.log: 2026-02-19T20:44:30.559 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80770.log 2026-02-19T20:44:30.560 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59650.log 2026-02-19T20:44:30.560 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.80770.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58838.log 2026-02-19T20:44:30.560 INFO:teuthology.orchestra.run.trial080.stderr: 92.2% 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.80770.log.gz 2026-02-19T20:44:30.560 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.62847.log.gz 2026-02-19T20:44:30.561 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83614.log: /var/log/ceph/cluster2-client.mirror.59650.log: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.59650.log.gz 2026-02-19T20:44:30.561 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose 87.6% -- -- replaced with /var/log/ceph/cluster2-client.mirror.83614.log.gz /var/log/ceph/cluster1-client.mirror.52009.log 2026-02-19T20:44:30.561 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.561 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58838.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72126.log 2026-02-19T20:44:30.562 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52009.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58838.log.gz 2026-02-19T20:44:30.562 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56286.log 2026-02-19T20:44:30.562 INFO:teuthology.orchestra.run.trial080.stderr: 87.8% -- replaced with /var/log/ceph/cluster1-client.mirror.52009.log.gz 2026-02-19T20:44:30.562 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72126.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82006.log 2026-02-19T20:44:30.563 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56286.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67844.log 2026-02-19T20:44:30.563 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56286.log.gz 2026-02-19T20:44:30.563 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82006.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55593.log 2026-02-19T20:44:30.563 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.72126.log.gz 2026-02-19T20:44:30.563 INFO:teuthology.orchestra.run.trial080.stderr: 86.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82006.log.gz 2026-02-19T20:44:30.563 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67844.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61608.log 2026-02-19T20:44:30.563 INFO:teuthology.orchestra.run.trial080.stderr: 87.7% -- replaced with /var/log/ceph/cluster1-client.mirror.67844.log.gz 2026-02-19T20:44:30.564 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61402.log 2026-02-19T20:44:30.564 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55593.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85819.log 2026-02-19T20:44:30.564 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61608.log: /var/log/ceph/cluster2-client.mirror.61402.log: 2.6% -- replaced with /var/log/ceph/cluster1-client.mirror.61608.log.gz 2026-02-19T20:44:30.564 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74215.log 2026-02-19T20:44:30.565 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.61402.log.gz 2026-02-19T20:44:30.565 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85819.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85819.log.gz 2026-02-19T20:44:30.568 INFO:teuthology.orchestra.run.trial080.stderr:gzip 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55593.log.gz 2026-02-19T20:44:30.568 INFO:teuthology.orchestra.run.trial080.stderr: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76767.log 2026-02-19T20:44:30.568 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74215.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83328.log 2026-02-19T20:44:30.568 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.74215.log.gz 2026-02-19T20:44:30.568 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76767.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71632.log 2026-02-19T20:44:30.568 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.76767.log.gz 2026-02-19T20:44:30.569 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83328.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38230.log 2026-02-19T20:44:30.569 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.83328.log.gz 2026-02-19T20:44:30.569 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71632.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75042.log 2026-02-19T20:44:30.569 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/ceph.tmp-client.admin.38230.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38230.log.gz 2026-02-19T20:44:30.570 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76829.log 2026-02-19T20:44:30.570 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75042.log: gzip -5 --verbose 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.71632.log.gz 2026-02-19T20:44:30.570 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster2-client.mirror.68911.log 2026-02-19T20:44:30.570 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.75042.log.gz 2026-02-19T20:44:30.571 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76829.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83075.log 2026-02-19T20:44:30.571 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.76829.log.gz 2026-02-19T20:44:30.571 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58235.log 2026-02-19T20:44:30.571 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68911.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84219.log 2026-02-19T20:44:30.571 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.68911.log.gz 2026-02-19T20:44:30.571 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83075.log: /var/log/ceph/cluster2-client.mirror.58235.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72013.log 2026-02-19T20:44:30.571 INFO:teuthology.orchestra.run.trial080.stderr: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.58235.log.gz 2026-02-19T20:44:30.572 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.83075.log.gz 2026-02-19T20:44:30.572 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51138.log 2026-02-19T20:44:30.572 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.84219.log: /var/log/ceph/cluster1-client.mirror.72013.log: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.84219.log.gz 2026-02-19T20:44:30.572 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60715.log 2026-02-19T20:44:30.573 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.72013.log.gz 2026-02-19T20:44:30.573 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51138.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83898.log 2026-02-19T20:44:30.573 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60715.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.60715.log.gz 2026-02-19T20:44:30.574 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80050.log 2026-02-19T20:44:30.574 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83898.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69341.log 2026-02-19T20:44:30.574 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.83898.log.gz 2026-02-19T20:44:30.574 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.80050.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65167.log 2026-02-19T20:44:30.574 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.80050.log.gz 2026-02-19T20:44:30.574 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77056.log 2026-02-19T20:44:30.574 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69341.log: /var/log/ceph/cluster2-client.mirror.65167.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58631.log 2026-02-19T20:44:30.575 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster2-client.mirror.65167.log.gz 2026-02-19T20:44:30.575 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.77056.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.69341.log.gz 2026-02-19T20:44:30.575 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.77056.log.gz 2026-02-19T20:44:30.575 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60118.log 2026-02-19T20:44:30.576 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58631.log: gzip -5 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.58631.log.gz 2026-02-19T20:44:30.576 INFO:teuthology.orchestra.run.trial080.stderr: --verbose -- /var/log/ceph/cluster1-client.mirror.82367.log 2026-02-19T20:44:30.576 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60118.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66679.log 2026-02-19T20:44:30.576 INFO:teuthology.orchestra.run.trial080.stderr: 88.5%/var/log/ceph/cluster1-client.mirror.82367.log: -- replaced with /var/log/ceph/cluster2-client.mirror.60118.log.gz 2026-02-19T20:44:30.576 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76994.log 2026-02-19T20:44:30.577 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.82367.log.gz 2026-02-19T20:44:30.577 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84512.log 2026-02-19T20:44:30.577 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66679.log: /var/log/ceph/cluster2-client.mirror.76994.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47441.log 2026-02-19T20:44:30.577 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76994.log.gz 2026-02-19T20:44:30.578 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.84512.log: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.84512.log.gz 2026-02-19T20:44:30.578 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74504.log 2026-02-19T20:44:30.578 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.66679.log.gz 2026-02-19T20:44:30.578 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.47441.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47441.log.gz 2026-02-19T20:44:30.578 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59162.log 2026-02-19T20:44:30.579 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74504.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84909.log 2026-02-19T20:44:30.579 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59162.log: 79.3%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57760.log 2026-02-19T20:44:30.579 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.59162.log.gz 2026-02-19T20:44:30.579 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.74504.log.gz 2026-02-19T20:44:30.580 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.84909.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84909.log.gz 2026-02-19T20:44:30.580 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76488.log 2026-02-19T20:44:30.581 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57760.log: 79.2%gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48464.log 2026-02-19T20:44:30.581 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.57760.log.gz 2026-02-19T20:44:30.581 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75000.log 2026-02-19T20:44:30.581 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76488.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77272.log 2026-02-19T20:44:30.582 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.48464.log: /var/log/ceph/cluster2-client.mirror.75000.log: 80.5% -- replaced with /var/log/ceph/cluster1-client.admin.48464.log.gz 2026-02-19T20:44:30.582 INFO:teuthology.orchestra.run.trial080.stderr: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76488.log.gz 2026-02-19T20:44:30.582 INFO:teuthology.orchestra.run.trial080.stderr: 90.0%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60938.log 2026-02-19T20:44:30.583 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.75000.log.gz 2026-02-19T20:44:30.583 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77272.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65263.log 2026-02-19T20:44:30.583 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77272.log.gz 2026-02-19T20:44:30.583 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60938.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50597.log 2026-02-19T20:44:30.584 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65263.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/cluster2-client.mirror.60938.log.gz -- 2026-02-19T20:44:30.584 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster2-client.mirror.63077.log 2026-02-19T20:44:30.584 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65263.log.gz 2026-02-19T20:44:30.584 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.50597.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55110.log 2026-02-19T20:44:30.585 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63077.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.50597.log.gz 2026-02-19T20:44:30.585 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81986.log 2026-02-19T20:44:30.585 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55110.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70319.log 2026-02-19T20:44:30.585 INFO:teuthology.orchestra.run.trial080.stderr: 89.5% -- replaced with /var/log/ceph/cluster1-client.mirror.55110.log.gz 2026-02-19T20:44:30.586 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81986.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.63077.log.gz 2026-02-19T20:44:30.586 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81986.log.gz 2026-02-19T20:44:30.586 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.53510.log 2026-02-19T20:44:30.586 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71543.log 2026-02-19T20:44:30.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70319.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.70319.log.gz 2026-02-19T20:44:30.587 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65350.log 2026-02-19T20:44:30.587 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.0.53510.log: /var/log/ceph/cluster1-client.mirror.71543.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75455.log 2026-02-19T20:44:30.587 INFO:teuthology.orchestra.run.trial080.stderr: 87.3%/var/log/ceph/cluster2-client.mirror.65350.log: -- replaced with /var/log/ceph/cluster1-client.mirror.71543.log.gz 2026-02-19T20:44:30.588 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.65350.log.gz 2026-02-19T20:44:30.588 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57012.log 2026-02-19T20:44:30.588 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75455.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78870.log 2026-02-19T20:44:30.589 INFO:teuthology.orchestra.run.trial080.stderr: 87.2%/var/log/ceph/cluster2-client.mirror.57012.log: -- replaced with /var/log/ceph/cluster2-client.mirror.75455.log.gz 2026-02-19T20:44:30.589 INFO:teuthology.orchestra.run.trial080.stderr: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.57012.log.gz 2026-02-19T20:44:30.589 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.47854.log 2026-02-19T20:44:30.589 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78870.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63346.log 2026-02-19T20:44:30.590 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78870.log.gz 2026-02-19T20:44:30.590 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.0.47854.log: 93.1% -- replaced with /var/log/ceph/cluster1-client.mirror.0.53510.log.gz 2026-02-19T20:44:30.590 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79432.log 2026-02-19T20:44:30.591 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63346.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.63346.log.gz 2026-02-19T20:44:30.592 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59847.log 2026-02-19T20:44:30.592 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79432.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70776.log 2026-02-19T20:44:30.592 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.79432.log.gz 2026-02-19T20:44:30.593 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59847.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59847.log.gz 2026-02-19T20:44:30.593 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71101.log 2026-02-19T20:44:30.593 INFO:teuthology.orchestra.run.trial080.stderr: 93.5% -- replaced with /var/log/ceph/cluster1-client.mirror.0.47854.log.gz 2026-02-19T20:44:30.593 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75062.log 2026-02-19T20:44:30.594 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70776.log: /var/log/ceph/cluster1-client.mirror.71101.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63434.log 2026-02-19T20:44:30.594 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.70776.log.gz 2026-02-19T20:44:30.594 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.71101.log.gz 2026-02-19T20:44:30.594 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75062.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85574.log 2026-02-19T20:44:30.595 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63434.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61566.log 2026-02-19T20:44:30.595 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.63434.log.gz/var/log/ceph/cluster2-client.admin.85574.log: 2026-02-19T20:44:30.595 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85574.log.gz 2026-02-19T20:44:30.595 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.75062.log.gz 2026-02-19T20:44:30.595 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55525.log 2026-02-19T20:44:30.596 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61566.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54842.log 2026-02-19T20:44:30.596 INFO:teuthology.orchestra.run.trial080.stderr: 1.3% -- replaced with /var/log/ceph/cluster2-client.mirror.61566.log.gz 2026-02-19T20:44:30.596 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55525.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81883.log 2026-02-19T20:44:30.597 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54842.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59554.log 2026-02-19T20:44:30.597 INFO:teuthology.orchestra.run.trial080.stderr: 89.1% -- replaced with /var/log/ceph/cluster1-client.mirror.54842.log.gz 2026-02-19T20:44:30.597 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57466.log.gz 2026-02-19T20:44:30.597 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81883.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54818.log 2026-02-19T20:44:30.598 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59554.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81883.log.gz 2026-02-19T20:44:30.598 INFO:teuthology.orchestra.run.trial080.stderr: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.59554.log.gz 2026-02-19T20:44:30.598 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68152.log 2026-02-19T20:44:30.598 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54818.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58648.log 2026-02-19T20:44:30.598 INFO:teuthology.orchestra.run.trial080.stderr: 89.0% -- replaced with /var/log/ceph/cluster1-client.mirror.54818.log.gz 2026-02-19T20:44:30.598 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68152.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63458.log 2026-02-19T20:44:30.599 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.68152.log.gz 2026-02-19T20:44:30.599 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79594.log 2026-02-19T20:44:30.599 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58648.log: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55525.log.gz 2026-02-19T20:44:30.599 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82720.log 2026-02-19T20:44:30.599 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63458.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58648.log.gz 2026-02-19T20:44:30.600 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79594.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78577.log 2026-02-19T20:44:30.600 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82720.log: 87.2% 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.79594.log.gz -- replaced with /var/log/ceph/cluster1-client.mirror.63458.log.gz 2026-02-19T20:44:30.600 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.600 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85866.log 2026-02-19T20:44:30.600 INFO:teuthology.orchestra.run.trial080.stderr: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.82720.log.gz 2026-02-19T20:44:30.600 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56386.log 2026-02-19T20:44:30.601 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78577.log: /var/log/ceph/cluster2-client.admin.85866.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78577.log.gz 2026-02-19T20:44:30.601 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- 0.0% /var/log/ceph/cluster2-client.mirror.82846.log -- replaced with /var/log/ceph/cluster2-client.admin.85866.log.gz 2026-02-19T20:44:30.601 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.601 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56386.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70343.log 2026-02-19T20:44:30.601 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.56386.log.gz/var/log/ceph/cluster2-client.mirror.82846.log: 2026-02-19T20:44:30.601 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70896.log 2026-02-19T20:44:30.602 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.82846.log.gz 2026-02-19T20:44:30.602 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82679.log 2026-02-19T20:44:30.602 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70343.log: /var/log/ceph/cluster1-client.mirror.70896.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59179.log 2026-02-19T20:44:30.602 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.70343.log.gz 2026-02-19T20:44:30.602 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.70896.log.gz 2026-02-19T20:44:30.602 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77970.log 2026-02-19T20:44:30.603 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82679.log: /var/log/ceph/cluster2-client.mirror.59179.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57128.log 2026-02-19T20:44:30.603 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.82679.log.gz 2026-02-19T20:44:30.603 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77970.log: 88.7%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81001.log 2026-02-19T20:44:30.603 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.59179.log.gz 2026-02-19T20:44:30.603 INFO:teuthology.orchestra.run.trial080.stderr: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.77970.log.gz 2026-02-19T20:44:30.604 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57128.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82576.log 2026-02-19T20:44:30.604 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.57128.log.gz 2026-02-19T20:44:30.604 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64618.log 2026-02-19T20:44:30.604 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81001.log: /var/log/ceph/cluster2-client.mirror.82576.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65999.log 2026-02-19T20:44:30.604 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.81001.log.gz 2026-02-19T20:44:30.605 INFO:teuthology.orchestra.run.trial080.stderr: 86.7% -- replaced with /var/log/ceph/cluster2-client.mirror.82576.log.gz 2026-02-19T20:44:30.605 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64618.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83716.log 2026-02-19T20:44:30.605 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65999.log: 88.3%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55224.log 2026-02-19T20:44:30.605 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.64618.log.gz 2026-02-19T20:44:30.605 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83716.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74256.log 2026-02-19T20:44:30.606 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.83716.log.gz 2026-02-19T20:44:30.606 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55376.log 2026-02-19T20:44:30.606 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55224.log: /var/log/ceph/cluster2-client.mirror.74256.log: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.65999.log.gz 2026-02-19T20:44:30.606 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79230.log 2026-02-19T20:44:30.607 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55376.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74256.log.gz 2026-02-19T20:44:30.609 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55224.log.gz 2026-02-19T20:44:30.609 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55376.log.gz 2026-02-19T20:44:30.609 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65790.log 2026-02-19T20:44:30.610 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59004.log 2026-02-19T20:44:30.610 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79230.log: /var/log/ceph/cluster1-client.mirror.65790.log: gzip -5 --verbose -- 86.4% /var/log/ceph/cluster2-client.mirror.52820.log 2026-02-19T20:44:30.610 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.79230.log.gz 2026-02-19T20:44:30.610 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59004.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75992.log 2026-02-19T20:44:30.611 INFO:teuthology.orchestra.run.trial080.stderr: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.59004.log.gz 2026-02-19T20:44:30.611 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52820.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78060.log 2026-02-19T20:44:30.611 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75992.log: 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.65790.log.gz 2026-02-19T20:44:30.611 INFO:teuthology.orchestra.run.trial080.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.52820.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71077.log 2026-02-19T20:44:30.611 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.612 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78060.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56406.log 2026-02-19T20:44:30.612 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78060.log.gz 2026-02-19T20:44:30.612 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71077.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50962.log 2026-02-19T20:44:30.613 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.71077.log.gz 2026-02-19T20:44:30.613 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.75992.log.gz 2026-02-19T20:44:30.613 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56406.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79320.log 2026-02-19T20:44:30.613 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.82210.log.gz 2026-02-19T20:44:30.613 INFO:teuthology.orchestra.run.trial080.stderr: 87.5%/var/log/ceph/cluster1-client.mirror.50962.log: -- replaced with /var/log/ceph/cluster2-client.mirror.56406.log.gz 2026-02-19T20:44:30.613 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.48634.log 2026-02-19T20:44:30.613 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.50962.log.gz 2026-02-19T20:44:30.613 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79320.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77295.log 2026-02-19T20:44:30.614 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.79320.log.gz 2026-02-19T20:44:30.614 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.48634.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63326.log 2026-02-19T20:44:30.614 INFO:teuthology.orchestra.run.trial080.stderr: 53.8% -- replaced with /var/log/ceph/cluster1-client.mirror.48634.log.gz 2026-02-19T20:44:30.614 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77295.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63828.log 2026-02-19T20:44:30.614 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.77295.log.gz 2026-02-19T20:44:30.614 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63326.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51482.log 2026-02-19T20:44:30.615 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63828.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71939.log 2026-02-19T20:44:30.615 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.51482.log: 92.4%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82515.log 2026-02-19T20:44:30.615 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.63326.log.gz 2026-02-19T20:44:30.616 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.51482.log.gz 2026-02-19T20:44:30.616 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71939.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53052.log 2026-02-19T20:44:30.616 INFO:teuthology.orchestra.run.trial080.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.63828.log.gz 2026-02-19T20:44:30.616 INFO:teuthology.orchestra.run.trial080.stderr: 88.9%/var/log/ceph/cluster2-client.mirror.82515.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71318.log 2026-02-19T20:44:30.616 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.71939.log.gz 2026-02-19T20:44:30.616 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.82515.log.gz 2026-02-19T20:44:30.616 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52680.log 2026-02-19T20:44:30.617 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53052.log: /var/log/ceph/cluster1-client.mirror.71318.log: 89.3% -- replaced with /var/log/ceph/cluster1-client.mirror.53052.log.gzgzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58510.log 2026-02-19T20:44:30.617 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.617 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.71318.log.gz 2026-02-19T20:44:30.617 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57170.log 2026-02-19T20:44:30.617 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52680.log: /var/log/ceph/cluster2-client.mirror.58510.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60774.log 2026-02-19T20:44:30.617 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.58510.log.gz 2026-02-19T20:44:30.618 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.57589.log 2026-02-19T20:44:30.618 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57170.log: 79.3%/var/log/ceph/cluster2-client.mirror.60774.log: -- replaced with /var/log/ceph/cluster2-client.mirror.57170.log.gz 2026-02-19T20:44:30.618 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63056.log 2026-02-19T20:44:30.618 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose --/var/log/ceph/cluster1-client.mirror.57589.log: /var/log/ceph/cluster2-client.admin.48976.log 2026-02-19T20:44:30.619 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63056.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68580.log 2026-02-19T20:44:30.619 INFO:teuthology.orchestra.run.trial080.stderr: 94.0% -- replaced with /var/log/ceph/cluster1-client.mirror.57589.log.gz 2026-02-19T20:44:30.619 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.63056.log.gz 2026-02-19T20:44:30.619 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.48976.log: 80.1% -- replaced with /var/log/ceph/cluster2-client.admin.48976.log.gz 2026-02-19T20:44:30.619 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80641.log 2026-02-19T20:44:30.620 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83350.log 2026-02-19T20:44:30.620 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68580.log: /var/log/ceph/cluster2-client.mirror.80641.log: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.80641.log.gz 2026-02-19T20:44:30.620 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79659.log 2026-02-19T20:44:30.621 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65308.log 2026-02-19T20:44:30.621 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83350.log: /var/log/ceph/cluster1-client.mirror.79659.log: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.68580.log.gz 2026-02-19T20:44:30.621 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.79659.log.gz 2026-02-19T20:44:30.624 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70937.log 2026-02-19T20:44:30.625 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65308.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82254.log 2026-02-19T20:44:30.626 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70937.log: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.65308.log.gz 2026-02-19T20:44:30.626 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.70937.log.gz 2026-02-19T20:44:30.626 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62758.log 2026-02-19T20:44:30.626 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82254.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64531.log 2026-02-19T20:44:30.626 INFO:teuthology.orchestra.run.trial080.stderr: 86.2% -- replaced with /var/log/ceph/cluster2-client.mirror.82254.log.gz 2026-02-19T20:44:30.627 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62758.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65382.log 2026-02-19T20:44:30.627 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.62758.log.gz 2026-02-19T20:44:30.627 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64531.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77722.log 2026-02-19T20:44:30.627 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.64531.log.gz 2026-02-19T20:44:30.627 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65382.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57839.log 2026-02-19T20:44:30.628 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.65382.log.gz/var/log/ceph/cluster1-client.mirror.77722.log: 2026-02-19T20:44:30.628 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77722.log.gz 2026-02-19T20:44:30.628 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75889.log 2026-02-19T20:44:30.628 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57839.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.47539.log 2026-02-19T20:44:30.628 INFO:teuthology.orchestra.run.trial080.stderr: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.57839.log.gz 2026-02-19T20:44:30.629 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75889.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49874.log 2026-02-19T20:44:30.629 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.75889.log.gz 2026-02-19T20:44:30.629 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.47539.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.47539.log.gz 2026-02-19T20:44:30.629 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63482.log 2026-02-19T20:44:30.630 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49874.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79387.log 2026-02-19T20:44:30.630 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63482.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.63482.log.gz 2026-02-19T20:44:30.630 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58215.log.gz 2026-02-19T20:44:30.631 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69156.log 2026-02-19T20:44:30.631 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79387.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55462.log 2026-02-19T20:44:30.631 INFO:teuthology.orchestra.run.trial080.stderr: 86.1% -- replaced with /var/log/ceph/cluster1-client.mirror.79387.log.gz/var/log/ceph/cluster2-client.mirror.69156.log: 2026-02-19T20:44:30.632 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75620.log 2026-02-19T20:44:30.632 INFO:teuthology.orchestra.run.trial080.stderr: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.69156.log.gz 2026-02-19T20:44:30.634 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55462.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77097.log 2026-02-19T20:44:30.635 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58453.log.gz 2026-02-19T20:44:30.635 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75620.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55462.log.gz 2026-02-19T20:44:30.635 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59889.log 2026-02-19T20:44:30.636 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.75620.log.gz 2026-02-19T20:44:30.636 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.77097.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77925.log 2026-02-19T20:44:30.636 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59889.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.48677.log 2026-02-19T20:44:30.636 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77925.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77925.log.gz 2026-02-19T20:44:30.637 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.77097.log.gz 2026-02-19T20:44:30.637 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66482.log 2026-02-19T20:44:30.637 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.48677.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.48677.log.gz 2026-02-19T20:44:30.637 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52189.log 2026-02-19T20:44:30.638 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79987.log 2026-02-19T20:44:30.639 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66482.log: /var/log/ceph/cluster1-client.mirror.52189.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.66482.log.gz 2026-02-19T20:44:30.639 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66077.log 2026-02-19T20:44:30.640 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79987.log: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.79987.log.gz 2026-02-19T20:44:30.640 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60643.log 2026-02-19T20:44:30.641 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66077.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.66077.log.gz 2026-02-19T20:44:30.641 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59937.log 2026-02-19T20:44:30.641 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.52189.log.gz 2026-02-19T20:44:30.642 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60643.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63233.log 2026-02-19T20:44:30.642 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59937.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60643.log.gz 2026-02-19T20:44:30.642 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57486.log 2026-02-19T20:44:30.642 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59937.log.gz 2026-02-19T20:44:30.643 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63233.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46807.log 2026-02-19T20:44:30.643 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57486.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.63233.log.gz 2026-02-19T20:44:30.643 INFO:teuthology.orchestra.run.trial080.stderr: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.57486.log.gz 2026-02-19T20:44:30.643 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68535.log 2026-02-19T20:44:30.644 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.2.80574.log 2026-02-19T20:44:30.644 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46807.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46807.log.gz 2026-02-19T20:44:30.644 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68535.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73377.log 2026-02-19T20:44:30.644 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.68535.log.gz 2026-02-19T20:44:30.645 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.2.80574.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69407.log 2026-02-19T20:44:30.645 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73377.log: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.73377.log.gz 2026-02-19T20:44:30.645 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74856.log 2026-02-19T20:44:30.646 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69407.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.69407.log.gz 2026-02-19T20:44:30.646 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79867.log 2026-02-19T20:44:30.647 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74856.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74856.log.gz 2026-02-19T20:44:30.647 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82737.log 2026-02-19T20:44:30.648 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79867.log: 93.2% -- replaced with /var/log/ceph/cluster2-client.mirror.2.80574.log.gz 2026-02-19T20:44:30.648 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.79867.log.gz 2026-02-19T20:44:30.648 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74298.log 2026-02-19T20:44:30.649 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82737.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54057.log 2026-02-19T20:44:30.649 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74298.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.82737.log.gz 2026-02-19T20:44:30.649 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74298.log.gz 2026-02-19T20:44:30.649 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69890.log 2026-02-19T20:44:30.650 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54057.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69299.log 2026-02-19T20:44:30.651 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69890.log: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.69890.log.gz 2026-02-19T20:44:30.651 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75393.log 2026-02-19T20:44:30.653 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69299.log: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.69299.log.gz 2026-02-19T20:44:30.653 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.54057.log.gz 2026-02-19T20:44:30.653 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74794.log 2026-02-19T20:44:30.653 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61010.log 2026-02-19T20:44:30.654 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75393.log: /var/log/ceph/cluster2-client.mirror.74794.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.75393.log.gz 2026-02-19T20:44:30.654 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74794.log.gz 2026-02-19T20:44:30.654 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63735.log 2026-02-19T20:44:30.654 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62345.log 2026-02-19T20:44:30.654 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61010.log: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.61010.log.gz 2026-02-19T20:44:30.655 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63735.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75104.log 2026-02-19T20:44:30.655 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.63735.log.gz 2026-02-19T20:44:30.655 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62345.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71965.log 2026-02-19T20:44:30.656 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75104.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.75104.log.gz 2026-02-19T20:44:30.656 INFO:teuthology.orchestra.run.trial080.stderr: 92.5% -- replaced with /var/log/ceph/cluster2-client.mirror.62345.log.gz 2026-02-19T20:44:30.657 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76054.log 2026-02-19T20:44:30.657 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71429.log 2026-02-19T20:44:30.657 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71965.log: /var/log/ceph/cluster2-client.mirror.76054.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.71965.log.gz 2026-02-19T20:44:30.658 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59041.log 2026-02-19T20:44:30.658 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71429.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76054.log.gz 2026-02-19T20:44:30.658 INFO:teuthology.orchestra.run.trial080.stderr: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.71429.log.gz 2026-02-19T20:44:30.658 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78937.log 2026-02-19T20:44:30.659 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59613.log 2026-02-19T20:44:30.659 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59041.log: /var/log/ceph/cluster1-client.mirror.78937.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59041.log.gz 2026-02-19T20:44:30.659 INFO:teuthology.orchestra.run.trial080.stderr: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78937.log.gz 2026-02-19T20:44:30.659 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76220.log 2026-02-19T20:44:30.660 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76344.log 2026-02-19T20:44:30.660 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59613.log: /var/log/ceph/cluster2-client.mirror.76220.log: 87.7% -- replaced with /var/log/ceph/cluster2-client.mirror.76220.log.gz 2026-02-19T20:44:30.667 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75786.log 2026-02-19T20:44:30.668 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76344.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.76344.log.gz 2026-02-19T20:44:30.668 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68600.log 2026-02-19T20:44:30.669 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75786.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75786.log.gz 2026-02-19T20:44:30.669 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76849.log 2026-02-19T20:44:30.670 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68600.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59413.log.gz 2026-02-19T20:44:30.670 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55722.log 2026-02-19T20:44:30.670 INFO:teuthology.orchestra.run.trial080.stderr: 92.2%/var/log/ceph/cluster2-client.mirror.76849.log: -- replaced with /var/log/ceph/cluster1-client.mirror.68600.log.gz 2026-02-19T20:44:30.671 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75641.log 2026-02-19T20:44:30.671 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55722.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.76849.log.gz 2026-02-19T20:44:30.671 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69057.log 2026-02-19T20:44:30.672 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75641.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.75641.log.gz 2026-02-19T20:44:30.673 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71387.log 2026-02-19T20:44:30.673 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69057.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.69057.log.gz 2026-02-19T20:44:30.674 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62365.log 2026-02-19T20:44:30.674 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71387.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.55722.log.gz 2026-02-19T20:44:30.675 INFO:teuthology.orchestra.run.trial080.stderr: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.71387.log.gz 2026-02-19T20:44:30.675 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64934.log 2026-02-19T20:44:30.675 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62365.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72479.log 2026-02-19T20:44:30.676 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64934.log: 87.1% -- replaced with /var/log/ceph/cluster2-client.mirror.64934.log.gz 2026-02-19T20:44:30.676 INFO:teuthology.orchestra.run.trial080.stderr: 92.2%gzip -- replaced with /var/log/ceph/cluster1-client.mirror.62365.log.gz -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52105.log 2026-02-19T20:44:30.676 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.677 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72479.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71342.log 2026-02-19T20:44:30.677 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52105.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.72479.log.gz 2026-02-19T20:44:30.677 INFO:teuthology.orchestra.run.trial080.stderr: 87.8% -- replaced with /var/log/ceph/cluster1-client.mirror.52105.log.gz 2026-02-19T20:44:30.678 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57108.log 2026-02-19T20:44:30.678 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58532.log 2026-02-19T20:44:30.678 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71342.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.71342.log.gz 2026-02-19T20:44:30.679 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57108.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.69078.log 2026-02-19T20:44:30.679 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58532.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57108.log.gz 2026-02-19T20:44:30.680 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59957.log 2026-02-19T20:44:30.681 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.69078.log: 92.2% -- replaced with /var/log/ceph/cluster2-client.mirror.69078.log.gz 2026-02-19T20:44:30.681 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49095.log 2026-02-19T20:44:30.682 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59957.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59957.log.gz 2026-02-19T20:44:30.683 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49129.log 2026-02-19T20:44:30.683 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49095.log: 64.6% -- replaced with /var/log/ceph/cluster2-client.mirror.49095.log.gz 2026-02-19T20:44:30.683 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70046.log 2026-02-19T20:44:30.684 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49129.log: 56.3% -- replaced with /var/log/ceph/cluster2-client.mirror.49129.log.gz 2026-02-19T20:44:30.684 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52081.log 2026-02-19T20:44:30.685 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70046.log: 87.0% -- replaced with /var/log/ceph/cluster1-client.mirror.70046.log.gz 2026-02-19T20:44:30.685 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83634.log 2026-02-19T20:44:30.686 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52081.log: 88.2% -- replaced with /var/log/ceph/cluster1-client.mirror.52081.log.gz 2026-02-19T20:44:30.686 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77407.log 2026-02-19T20:44:30.687 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83634.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.83634.log.gz 2026-02-19T20:44:30.687 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57819.log 2026-02-19T20:44:30.688 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77407.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.77407.log.gz 2026-02-19T20:44:30.688 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62983.log 2026-02-19T20:44:30.706 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57819.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74070.log 2026-02-19T20:44:30.706 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62983.log: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.62983.log.gz 2026-02-19T20:44:30.719 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51051.log 2026-02-19T20:44:30.733 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74070.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74070.log.gz 2026-02-19T20:44:30.734 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52780.log 2026-02-19T20:44:30.744 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.51051.log: 93.3%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73966.log 2026-02-19T20:44:30.744 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52780.log: 89.4% -- replaced with /var/log/ceph/cluster2-client.mirror.52780.log.gz 2026-02-19T20:44:30.749 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.51051.log.gz 2026-02-19T20:44:30.753 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79117.log 2026-02-19T20:44:30.762 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73966.log: 87.4%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60921.log 2026-02-19T20:44:30.765 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79117.log: 86.5% -- replaced with /var/log/ceph/cluster2-client.mirror.73966.log.gz 2026-02-19T20:44:30.770 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.77015.log 2026-02-19T20:44:30.772 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60921.log: 79.2% -- replaced with /var/log/ceph/cluster1-client.mirror.79117.log.gz 2026-02-19T20:44:30.780 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81293.log 2026-02-19T20:44:30.781 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.77015.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.77015.log.gz 2026-02-19T20:44:30.782 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57545.log.gz 2026-02-19T20:44:30.786 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50890.log 2026-02-19T20:44:30.786 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.81293.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.81293.log.gz 2026-02-19T20:44:30.786 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53185.log 2026-02-19T20:44:30.787 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.50890.log: -- replaced with /var/log/ceph/cluster2-client.mirror.60921.log.gz 2026-02-19T20:44:30.787 INFO:teuthology.orchestra.run.trial080.stderr: 87.7% -- replaced with /var/log/ceph/cluster2-client.mirror.50890.log.gz 2026-02-19T20:44:30.787 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68065.log 2026-02-19T20:44:30.787 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53185.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.53185.log.gz 2026-02-19T20:44:30.787 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54487.log 2026-02-19T20:44:30.787 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68065.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68065.log.gz 2026-02-19T20:44:30.787 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66586.log 2026-02-19T20:44:30.789 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54487.log: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.54487.log.gz 2026-02-19T20:44:30.792 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85672.log 2026-02-19T20:44:30.792 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66586.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.66586.log.gz 2026-02-19T20:44:30.792 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83034.log 2026-02-19T20:44:30.792 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85672.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85672.log.gz 2026-02-19T20:44:30.792 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.56809.log.gz 2026-02-19T20:44:30.792 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51074.log 2026-02-19T20:44:30.792 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83034.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.83034.log.gz 2026-02-19T20:44:30.794 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79185.log 2026-02-19T20:44:30.794 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51410.log 2026-02-19T20:44:30.794 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51074.log: /var/log/ceph/cluster1-client.mirror.79185.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.79185.log.gz 2026-02-19T20:44:30.796 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.51074.log.gz 2026-02-19T20:44:30.801 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59571.log 2026-02-19T20:44:30.801 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.51410.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80285.log 2026-02-19T20:44:30.801 INFO:teuthology.orchestra.run.trial080.stderr: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.51410.log.gz 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59571.log: 88.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59571.log.gz 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82556.log 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80285.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57150.log 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.80285.log.gz 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82556.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62710.log 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57150.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.82556.log.gz 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.71003.log 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62710.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62710.log.gz 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82494.log 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.71003.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.71003.log.gz 2026-02-19T20:44:30.802 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73875.log 2026-02-19T20:44:30.804 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82494.log: 86.9% -- replaced with /var/log/ceph/cluster2-client.mirror.82494.log.gz 2026-02-19T20:44:30.805 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.83120.log 2026-02-19T20:44:30.805 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73875.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.73875.log.gz 2026-02-19T20:44:30.805 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71366.log 2026-02-19T20:44:30.806 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.0.83120.log: 92.9% -- replaced with /var/log/ceph/cluster2-client.mirror.0.83120.log.gz 2026-02-19T20:44:30.806 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.71519.log 2026-02-19T20:44:30.809 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71366.log: 87.8% -- replaced with /var/log/ceph/cluster1-client.mirror.71366.log.gz 2026-02-19T20:44:30.809 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76323.log 2026-02-19T20:44:30.809 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.71519.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.71519.log.gz 2026-02-19T20:44:30.809 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81142.log 2026-02-19T20:44:30.810 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76323.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76323.log.gz 2026-02-19T20:44:30.810 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74007.log 2026-02-19T20:44:30.811 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81142.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.81142.log.gz 2026-02-19T20:44:30.811 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81104.log 2026-02-19T20:44:30.811 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74007.log: 90.1% -- replaced with /var/log/ceph/cluster2-client.mirror.74007.log.gz 2026-02-19T20:44:30.812 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72397.log 2026-02-19T20:44:30.815 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81104.log: 90.7% -- replaced with /var/log/ceph/cluster2-client.mirror.81104.log.gz 2026-02-19T20:44:30.815 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52126.log 2026-02-19T20:44:30.815 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72397.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.72397.log.gz 2026-02-19T20:44:30.815 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54531.log 2026-02-19T20:44:30.818 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52126.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.52126.log.gz 2026-02-19T20:44:30.819 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81228.log 2026-02-19T20:44:30.827 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54531.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83778.log 2026-02-19T20:44:30.840 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.81228.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.81228.log.gz 2026-02-19T20:44:30.840 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.60685.log.gz 2026-02-19T20:44:30.841 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64970.log 2026-02-19T20:44:30.842 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83778.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.83778.log.gz 2026-02-19T20:44:30.850 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.56509.log 2026-02-19T20:44:30.850 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64970.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.64970.log.gz 2026-02-19T20:44:30.853 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69230.log 2026-02-19T20:44:30.853 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.56509.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.56509.log.gz 2026-02-19T20:44:30.853 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.59265.log 2026-02-19T20:44:30.853 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69230.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.69230.log.gz 2026-02-19T20:44:30.853 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70247.log 2026-02-19T20:44:30.856 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.59265.log: 95.2% -- replaced with /var/log/ceph/cluster1-client.mirror.59265.log.gz 2026-02-19T20:44:30.856 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65239.log 2026-02-19T20:44:30.858 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70247.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.70247.log.gz 2026-02-19T20:44:30.858 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67615.log 2026-02-19T20:44:30.859 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65239.log: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.65239.log.gz 2026-02-19T20:44:30.859 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50824.log 2026-02-19T20:44:30.859 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67615.log: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.67615.log.gz 2026-02-19T20:44:30.862 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77317.log 2026-02-19T20:44:30.862 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.50824.log: 89.3% -- replaced with /var/log/ceph/cluster2-client.mirror.50824.log.gz 2026-02-19T20:44:30.862 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67085.log 2026-02-19T20:44:30.862 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77317.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77317.log.gz 2026-02-19T20:44:30.862 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56787.log 2026-02-19T20:44:30.863 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67085.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.67085.log.gz 2026-02-19T20:44:30.863 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57898.log 2026-02-19T20:44:30.864 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56787.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.56787.log.gz 2026-02-19T20:44:30.864 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67519.log 2026-02-19T20:44:30.872 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57898.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76870.log 2026-02-19T20:44:30.885 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.67519.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.67519.log.gz 2026-02-19T20:44:30.886 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55937.log 2026-02-19T20:44:30.886 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76870.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76870.log.gz 2026-02-19T20:44:30.893 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.51571.log 2026-02-19T20:44:30.906 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55937.log: 93.4%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61382.log 2026-02-19T20:44:30.906 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.55937.log.gz 2026-02-19T20:44:30.909 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.51571.log: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.51571.log.gz 2026-02-19T20:44:30.922 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64409.log 2026-02-19T20:44:30.923 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61382.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61382.log.gz 2026-02-19T20:44:30.939 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84071.log 2026-02-19T20:44:30.939 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64409.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.64409.log.gz 2026-02-19T20:44:30.942 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58373.log.gz 2026-02-19T20:44:30.944 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72842.log 2026-02-19T20:44:30.944 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84071.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84071.log.gz 2026-02-19T20:44:30.944 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72821.log 2026-02-19T20:44:30.948 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72842.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.72842.log.gz 2026-02-19T20:44:30.948 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77902.log 2026-02-19T20:44:30.948 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72821.log: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.72821.log.gz 2026-02-19T20:44:30.948 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82168.log 2026-02-19T20:44:30.948 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77902.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.77902.log.gz 2026-02-19T20:44:30.948 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60074.log 2026-02-19T20:44:30.949 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82168.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.82168.log.gz 2026-02-19T20:44:30.949 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46706.log 2026-02-19T20:44:30.960 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60074.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.60160.log.gz 2026-02-19T20:44:30.960 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72884.log 2026-02-19T20:44:30.960 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46706.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46706.log.gzgzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62071.log 2026-02-19T20:44:30.960 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:30.960 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72884.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.72884.log.gz 2026-02-19T20:44:30.960 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.52504.log 2026-02-19T20:44:30.960 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62071.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62071.log.gz 2026-02-19T20:44:30.962 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53143.log 2026-02-19T20:44:30.962 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.52504.log: 88.2% -- replaced with /var/log/ceph/cluster1-client.mirror.52504.log.gz 2026-02-19T20:44:30.962 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.85427.log 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53143.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.53143.log.gz 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56557.log 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.85427.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.85427.log.gz 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77655.log 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56557.log: 79.3% -- replaced with /var/log/ceph/cluster2-client.mirror.56557.log.gz 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84683.log 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77655.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.77655.log.gz 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83696.log 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84683.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.84683.log.gz 2026-02-19T20:44:30.969 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62959.log 2026-02-19T20:44:30.972 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83696.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.83696.log.gz 2026-02-19T20:44:30.972 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54099.log 2026-02-19T20:44:30.973 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62959.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62959.log.gz 2026-02-19T20:44:30.973 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84491.log 2026-02-19T20:44:30.973 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54099.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.54099.log.gz 2026-02-19T20:44:30.973 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.1.48095.log 2026-02-19T20:44:30.981 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84491.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68306.log 2026-02-19T20:44:30.982 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.1.48095.log: 93.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59889.log.gz 2026-02-19T20:44:30.987 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57387.log.gz 2026-02-19T20:44:30.988 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63185.log 2026-02-19T20:44:30.988 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68306.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68306.log.gz 2026-02-19T20:44:30.996 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70406.log 2026-02-19T20:44:30.996 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63185.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.63185.log.gz 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.72188.log 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70406.log: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.1.48095.log.gz 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.70406.log.gz 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61276.log 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.72188.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58782.log 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.72188.log.gz 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61276.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84577.log 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58782.log: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.61276.log.gz 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79275.log 2026-02-19T20:44:31.004 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.84577.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.84577.log.gz 2026-02-19T20:44:31.005 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64800.log 2026-02-19T20:44:31.005 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79275.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.79275.log.gz 2026-02-19T20:44:31.005 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77185.log 2026-02-19T20:44:31.005 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.64800.log: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.64800.log.gz 2026-02-19T20:44:31.005 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72375.log 2026-02-19T20:44:31.005 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77185.log: 1.3% -- replaced with /var/log/ceph/cluster1-client.mirror.77185.log.gz 2026-02-19T20:44:31.005 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75538.log 2026-02-19T20:44:31.008 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72375.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.72375.log.gz 2026-02-19T20:44:31.008 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61456.log 2026-02-19T20:44:31.008 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75538.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75538.log.gz 2026-02-19T20:44:31.008 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59709.log 2026-02-19T20:44:31.008 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61456.log: 79.1% -- replaced with /var/log/ceph/cluster2-client.mirror.61456.log.gz 2026-02-19T20:44:31.008 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78330.log 2026-02-19T20:44:31.016 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59709.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77206.log 2026-02-19T20:44:31.029 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78330.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78330.log.gz 2026-02-19T20:44:31.033 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80960.log 2026-02-19T20:44:31.034 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77206.log: 1.3% -- replaced with /var/log/ceph/cluster1-client.mirror.77206.log.gz 2026-02-19T20:44:31.040 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59221.log 2026-02-19T20:44:31.049 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80960.log: 87.6% 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.61156.log.gz 2026-02-19T20:44:31.049 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.68833.log 2026-02-19T20:44:31.054 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59221.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57071.log.gz 2026-02-19T20:44:31.054 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.80960.log.gz 2026-02-19T20:44:31.054 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83878.log 2026-02-19T20:44:31.055 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.68833.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.68833.log.gz 2026-02-19T20:44:31.055 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77475.log 2026-02-19T20:44:31.056 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83878.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.83878.log.gz 2026-02-19T20:44:31.059 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78082.log 2026-02-19T20:44:31.059 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77475.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77475.log.gz 2026-02-19T20:44:31.059 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61335.log 2026-02-19T20:44:31.060 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.79573.log.gz 2026-02-19T20:44:31.060 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78082.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78082.log.gz 2026-02-19T20:44:31.060 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57444.log 2026-02-19T20:44:31.060 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61335.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73532.log 2026-02-19T20:44:31.060 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57444.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.57444.log.gz 2026-02-19T20:44:31.060 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80389.log 2026-02-19T20:44:31.063 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73532.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.73532.log.gz 2026-02-19T20:44:31.063 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68853.log 2026-02-19T20:44:31.063 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80389.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.80389.log.gz 2026-02-19T20:44:31.063 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75145.log 2026-02-19T20:44:31.064 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68853.log: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.68853.log.gz 2026-02-19T20:44:31.064 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53727.log 2026-02-19T20:44:31.065 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75145.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.75145.log.gz 2026-02-19T20:44:31.065 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48399.log 2026-02-19T20:44:31.073 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53727.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60015.log 2026-02-19T20:44:31.081 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.48399.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.54531.log.gz 2026-02-19T20:44:31.084 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56750.log 2026-02-19T20:44:31.084 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60015.log: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60015.log.gz 2026-02-19T20:44:31.084 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.48399.log.gz 2026-02-19T20:44:31.085 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.3.48158.log 2026-02-19T20:44:31.085 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56750.log: 79.2% -- replaced with /var/log/ceph/cluster2-client.mirror.56750.log.gz 2026-02-19T20:44:31.085 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57150.log.gz 2026-02-19T20:44:31.085 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50986.log 2026-02-19T20:44:31.085 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.3.48158.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.83941.log 2026-02-19T20:44:31.085 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50986.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.50986.log.gz 2026-02-19T20:44:31.085 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78600.log 2026-02-19T20:44:31.089 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.83941.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.83941.log.gz 2026-02-19T20:44:31.089 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80264.log 2026-02-19T20:44:31.089 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78600.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78600.log.gz 2026-02-19T20:44:31.089 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.50282.log 2026-02-19T20:44:31.089 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.80264.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.80264.log.gz 2026-02-19T20:44:31.089 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62574.log 2026-02-19T20:44:31.089 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.50282.log: 87.1% -- replaced with /var/log/ceph/cluster1-client.admin.50282.log.gz 2026-02-19T20:44:31.090 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62529.log 2026-02-19T20:44:31.092 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62574.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.62574.log.gz 2026-02-19T20:44:31.092 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84449.log 2026-02-19T20:44:31.093 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62529.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.62529.log.gz 2026-02-19T20:44:31.093 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58490.log 2026-02-19T20:44:31.093 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84449.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.84449.log.gz 2026-02-19T20:44:31.093 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84820.log 2026-02-19T20:44:31.096 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58490.log: 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.58490.log.gz 2026-02-19T20:44:31.096 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67469.log 2026-02-19T20:44:31.096 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84820.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84820.log.gz 2026-02-19T20:44:31.096 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68017.log 2026-02-19T20:44:31.096 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67469.log: 88.8% -- replaced with /var/log/ceph/cluster2-client.mirror.67469.log.gz 2026-02-19T20:44:31.097 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70917.log 2026-02-19T20:44:31.097 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68017.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68017.log.gz 2026-02-19T20:44:31.098 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84703.log 2026-02-19T20:44:31.100 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70917.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.70917.log.gz 2026-02-19T20:44:31.100 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.0.59265.log 2026-02-19T20:44:31.100 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84703.log: 86.9% -- replaced with /var/log/ceph/cluster2-client.mirror.84703.log.gz 2026-02-19T20:44:31.100 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63305.log 2026-02-19T20:44:31.109 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.0.59265.log: gzip -5 --verbose -- 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57977.log.gz 2026-02-19T20:44:31.109 INFO:teuthology.orchestra.run.trial080.stderr: /var/log/ceph/cluster2-client.mirror.74732.log 2026-02-19T20:44:31.110 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63305.log: 87.5%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57229.log 2026-02-19T20:44:31.111 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74732.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74732.log.gz 2026-02-19T20:44:31.111 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.63305.log.gz 2026-02-19T20:44:31.111 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62553.log 2026-02-19T20:44:31.132 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57229.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65480.log 2026-02-19T20:44:31.133 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62553.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62553.log.gz 2026-02-19T20:44:31.143 INFO:teuthology.orchestra.run.trial080.stderr: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.0.59265.log.gz 2026-02-19T20:44:31.170 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79724.log 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65480.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65480.log.gz 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr: 93.8%gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81774.log 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.3.48158.log.gz 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79724.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76571.log 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.81774.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.79724.log.gz 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.mirror.81774.log.gz 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64435.log 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76571.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.79847.log 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76571.log.gz 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64435.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.50771.log 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.79847.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.64435.log.gz 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.79847.log.gz 2026-02-19T20:44:31.171 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82130.log 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.50771.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72551.log 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.51138.log.gz 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr: 87.1% -- replaced with /var/log/ceph/cluster1-client.admin.50771.log.gz 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82130.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.56532.log 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.72551.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60528.log 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster1-client.mirror.72551.log.gz 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.56532.log: 90.7% -- replaced with /var/log/ceph/cluster2-client.mirror.82130.log.gz 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.56532.log.gz 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76385.log 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60528.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49078.log 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76385.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65141.log 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76385.log.gz 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49078.log: 64.3% -- replaced with /var/log/ceph/cluster2-client.mirror.49078.log.gz 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66864.log 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65141.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80306.log 2026-02-19T20:44:31.172 INFO:teuthology.orchestra.run.trial080.stderr: 88.9%/var/log/ceph/cluster1-client.mirror.66864.log: -- replaced with /var/log/ceph/cluster2-client.mirror.65141.log.gz 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.66864.log.gz 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62252.log 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose --/var/log/ceph/cluster2-client.mirror.80306.log: /var/log/ceph/cluster2-client.mirror.76282.log 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.80306.log.gz 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62252.log: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.62252.log.gz 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65408.log 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76282.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.64257.log 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.76282.log.gz 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65408.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.65408.log.gz 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77947.log 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose/var/log/ceph/cluster1-client.mirror.64257.log: -- /var/log/ceph/cluster2-client.mirror.73987.log 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.56984.log.gz 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.64257.log.gz 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77947.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.77947.log.gz 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53414.log 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73987.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69684.log 2026-02-19T20:44:31.173 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53414.log: 87.7% -- replaced with /var/log/ceph/cluster2-client.mirror.73987.log.gz 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58740.log 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr: 88.8% -- replaced with /var/log/ceph/cluster1-client.mirror.53414.log.gz 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66199.log 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.69684.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75910.log 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.69684.log.gz 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58740.log: /var/log/ceph/cluster1-client.mirror.66199.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.66199.log.gz 88.5% -- replaced with /var/log/ceph/cluster2-client.mirror.58740.log.gz 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.82779.log 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75910.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66888.log 87.6% 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster2-client.mirror.75910.log.gz 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.82779.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75848.log 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66888.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55656.log 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.66888.log.gz 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75848.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75848.log.gz 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78397.log 2026-02-19T20:44:31.174 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55656.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.54444.log 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78397.log: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.78397.log.gz 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73305.log 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.54444.log: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55656.log.gz 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59391.log 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73305.log: 87.0% -- replaced with /var/log/ceph/cluster2-client.mirror.73305.log.gz 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.54444.log.gz 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.67748.log 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59391.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.63575.log 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59391.log.gz 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.67748.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81540.log 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.63575.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.67748.log.gz 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63595.log 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81540.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81540.log.gz 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.50185.log 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr: 92.3% -- replaced with /var/log/ceph/cluster2-client.mirror.63575.log.gz 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63595.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65432.log 2026-02-19T20:44:31.175 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.50185.log: 74.4% -- replaced with /var/log/ceph/cluster1-client.mirror.50185.log.gz 2026-02-19T20:44:31.176 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77497.log 2026-02-19T20:44:31.176 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65432.log: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65432.log.gz 2026-02-19T20:44:31.176 INFO:teuthology.orchestra.run.trial080.stderr: 92.1% -- replaced with /var/log/ceph/cluster1-client.mirror.63595.log.gz 2026-02-19T20:44:31.176 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.52573.log 2026-02-19T20:44:31.176 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77497.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74525.log 2026-02-19T20:44:31.176 INFO:teuthology.orchestra.run.trial080.stderr: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.77497.log.gz 2026-02-19T20:44:31.176 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.52573.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.0.56661.log 2026-02-19T20:44:31.176 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74525.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.74525.log.gz 2026-02-19T20:44:31.176 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.66504.log 2026-02-19T20:44:31.177 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.0.56661.log: 93.4% -- replaced with /var/log/ceph/cluster2-client.mirror.52573.log.gz 2026-02-19T20:44:31.177 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59142.log 2026-02-19T20:44:31.177 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.66504.log: 88.3% -- replaced with /var/log/ceph/cluster2-client.mirror.66504.log.gz 2026-02-19T20:44:31.177 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.83209.log 2026-02-19T20:44:31.178 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59142.log: 93.6% -- replaced with /var/log/ceph/cluster2-client.mirror.60528.log.gz 2026-02-19T20:44:31.179 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.64907.log 2026-02-19T20:44:31.179 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.83209.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.83209.log.gz 2026-02-19T20:44:31.180 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.46469.log 2026-02-19T20:44:31.180 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.64907.log: 88.9% -- replaced with /var/log/ceph/cluster2-client.mirror.64907.log.gz 2026-02-19T20:44:31.182 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55310.log 2026-02-19T20:44:31.182 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.46469.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.46469.log.gz 2026-02-19T20:44:31.182 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60201.log 2026-02-19T20:44:31.186 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55310.log: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.55310.log.gz 2026-02-19T20:44:31.187 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73239.log 2026-02-19T20:44:31.187 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60201.log: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.60201.log.gz 2026-02-19T20:44:31.187 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61515.log 2026-02-19T20:44:31.188 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73239.log: 88.2% -- replaced with /var/log/ceph/cluster2-client.mirror.73239.log.gz 2026-02-19T20:44:31.188 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75166.log 2026-02-19T20:44:31.192 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61515.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.60373.log.gz 2026-02-19T20:44:31.193 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59764.log 2026-02-19T20:44:31.194 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75166.log: 87.7% -- replaced with /var/log/ceph/cluster2-client.mirror.75166.log.gz 2026-02-19T20:44:31.194 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47686.log 2026-02-19T20:44:31.195 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59764.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.59764.log.gz 2026-02-19T20:44:31.195 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61710.log 2026-02-19T20:44:31.195 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47686.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47686.log.gz 2026-02-19T20:44:31.197 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77835.log 2026-02-19T20:44:31.197 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61710.log: 87.1% -- replaced with /var/log/ceph/cluster2-client.mirror.61710.log.gz 2026-02-19T20:44:31.197 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.59063.log 2026-02-19T20:44:31.198 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77835.log: 86.5% -- replaced with /var/log/ceph/cluster1-client.mirror.77835.log.gz 2026-02-19T20:44:31.198 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.admin.47245.log 2026-02-19T20:44:31.211 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.59063.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.55785.log 2026-02-19T20:44:31.211 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.admin.47245.log: 0.0% -- replaced with /var/log/ceph/cluster2-client.admin.47245.log.gz 2026-02-19T20:44:31.217 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50619.log 2026-02-19T20:44:31.220 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.55785.log: 93.3% -- replaced with /var/log/ceph/cluster1-client.mirror.0.56661.log.gz 2026-02-19T20:44:31.220 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.54561.log 2026-02-19T20:44:31.229 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.50619.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56126.log 2026-02-19T20:44:31.230 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.54561.log: 88.0% -- replaced with /var/log/ceph/cluster1-client.admin.54561.log.gz 2026-02-19T20:44:31.232 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster1-client.mirror.55785.log.gz 2026-02-19T20:44:31.250 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.53705.log 2026-02-19T20:44:31.251 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56126.log: 88.0% -- replaced with /var/log/ceph/cluster2-client.mirror.56126.log.gz 2026-02-19T20:44:31.258 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59613.log.gz 2026-02-19T20:44:31.260 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75269.log 2026-02-19T20:44:31.260 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.53705.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.53705.log.gz 2026-02-19T20:44:31.261 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57777.log 2026-02-19T20:44:31.261 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75269.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.75269.log.gz 2026-02-19T20:44:31.262 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.50798.log 2026-02-19T20:44:31.263 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57777.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.57777.log.gz 2026-02-19T20:44:31.263 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.57187.log 2026-02-19T20:44:31.264 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.50798.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.admin.50798.log.gz 2026-02-19T20:44:31.264 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60896.log 2026-02-19T20:44:31.265 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.57187.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.57187.log.gz 2026-02-19T20:44:31.266 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81965.log 2026-02-19T20:44:31.275 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60896.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74442.log 2026-02-19T20:44:31.287 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81965.log: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81965.log.gz 2026-02-19T20:44:31.292 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.70271.log 2026-02-19T20:44:31.292 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74442.log: 90.0% -- replaced with /var/log/ceph/cluster2-client.mirror.74442.log.gz 2026-02-19T20:44:31.297 INFO:teuthology.orchestra.run.trial080.stderr: 90.0% -- replaced with /var/log/ceph/cluster2-mon.a.log.gz 2026-02-19T20:44:31.300 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76530.log 2026-02-19T20:44:31.301 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.70271.log: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.70271.log.gz 2026-02-19T20:44:31.301 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.66240.log 2026-02-19T20:44:31.301 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76530.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.76530.log.gz 2026-02-19T20:44:31.301 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.79027.log 2026-02-19T20:44:31.301 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.66240.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.60774.log.gz 2026-02-19T20:44:31.305 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.66240.log.gz 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr: -- /var/log/ceph/cluster1-client.mirror.78892.log 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.79027.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.79027.log.gz 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.51551.log 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78780.log 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78892.log: /var/log/ceph/cluster2-client.mirror.51551.log: 86.3% -- replaced with /var/log/ceph/cluster1-client.mirror.78892.log.gz 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81417.log 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78780.log: 86.6% -- replaced with /var/log/ceph/cluster1-client.mirror.78780.log.gz 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.76615.log 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81417.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.81417.log.gz 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80073.log 2026-02-19T20:44:31.306 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.76615.log: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.51551.log.gz 2026-02-19T20:44:31.307 INFO:teuthology.orchestra.run.trial080.stderr: 89.0% -- replaced with /var/log/ceph/cluster1-client.mirror.76615.log.gz 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58569.log 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.80073.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.56767.log 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.80073.log.gz 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58569.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.62023.log 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.56767.log: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58532.log.gz 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58569.log.gz 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.56767.log.gz 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81355.log 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.62023.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.85103.log 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.62023.log.gz 2026-02-19T20:44:31.315 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81355.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.49168.log 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.81355.log.gz 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.85103.log: 0.0% -- replaced with /var/log/ceph/cluster1-client.admin.85103.log.gz 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.84934.log 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.50335.log 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.49168.log: /var/log/ceph/cluster1-client.mirror.84934.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84112.log 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr: 79.4% -- replaced with /var/log/ceph/cluster2-client.mirror.49168.log.gz 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /var/log/ceph/cluster1-client.mirror.84934.log.gz 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.50335.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.1.54729.log 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr:gzip/var/log/ceph/cluster2-client.mirror.84112.log: -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80878.log 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr: 89.2% -- replaced with /var/log/ceph/cluster2-client.mirror.50335.log.gz 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84112.log.gz 2026-02-19T20:44:31.316 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.1.54729.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.70364.log 2026-02-19T20:44:31.317 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80878.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74174.log 2026-02-19T20:44:31.317 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.80878.log.gz 2026-02-19T20:44:31.317 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.70364.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.70364.log.gz 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.61365.log 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74174.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.81797.log 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.74174.log.gz 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.61365.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.80980.log 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr: 79.5%/var/log/ceph/cluster1-client.mirror.81797.log: -- replaced with /var/log/ceph/cluster2-client.mirror.61365.log.gz 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr: 87.2% -- replaced with /var/log/ceph/cluster1-client.mirror.81797.log.gz 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.80574.log 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.53897.log 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.80980.log: /var/log/ceph/cluster1-client.mirror.80574.log: 82.7% -- replaced with /var/log/ceph/cluster1-client.mirror.80574.log.gz 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr: 86.8% -- replaced with /var/log/ceph/cluster2-client.mirror.80980.log.gz 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.61806.log 2026-02-19T20:44:31.329 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.53897.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81084.log 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr: 88.8% -- replaced with /var/log/ceph/cluster1-client.mirror.53897.log.gz 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.61806.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.admin.48481.log 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.61806.log.gz 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81084.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.78735.log 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.admin.48481.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.81084.log.gz 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr: 71.0% -- replaced with /var/log/ceph/cluster1-client.admin.48481.log.gz 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.72286.log 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.67656.log 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.78735.log: /var/log/ceph/cluster1-client.mirror.72286.log: 86.4% -- replaced with /var/log/ceph/cluster1-client.mirror.78735.log.gz 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.72286.log.gz 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-osd.2.log 2026-02-19T20:44:31.330 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84861.log 2026-02-19T20:44:31.334 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-osd.2.log: /var/log/ceph/cluster1-client.mirror.67656.log: 92.2% -- replaced with /var/log/ceph/cluster1-client.mirror.67656.log.gz 2026-02-19T20:44:31.334 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.81042.log 2026-02-19T20:44:31.334 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84861.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84861.log.gz 2026-02-19T20:44:31.334 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.62457.log 2026-02-19T20:44:31.334 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.81042.log: 87.4% -- replaced with /var/log/ceph/cluster2-client.mirror.81042.log.gz 2026-02-19T20:44:31.334 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.54424.log 2026-02-19T20:44:31.334 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.62457.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.62457.log.gz 2026-02-19T20:44:31.334 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.69033.log 2026-02-19T20:44:31.343 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.54424.log: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.54424.log.gz 2026-02-19T20:44:31.343 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.63960.log 2026-02-19T20:44:31.343 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57898.log.gz/var/log/ceph/cluster1-client.mirror.69033.log: 2026-02-19T20:44:31.343 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.69033.log.gz 2026-02-19T20:44:31.343 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65042.log 2026-02-19T20:44:31.343 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.63960.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.65328.log 2026-02-19T20:44:31.343 INFO:teuthology.orchestra.run.trial080.stderr: 87.4% -- replaced with /var/log/ceph/cluster1-client.mirror.63960.log.gz 2026-02-19T20:44:31.343 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.65042.log: gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68089.log 2026-02-19T20:44:31.343 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.65042.log.gz/var/log/ceph/cluster1-client.mirror.65328.log: 2026-02-19T20:44:31.344 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.60474.log 2026-02-19T20:44:31.344 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68089.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.65328.log.gz 2026-02-19T20:44:31.344 INFO:teuthology.orchestra.run.trial080.stderr: 87.5% -- replaced with /var/log/ceph/cluster1-client.mirror.68089.log.gz 2026-02-19T20:44:31.344 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.73940.log 2026-02-19T20:44:31.344 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.60474.log: gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.74463.log 2026-02-19T20:44:31.344 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.73940.log: 92.3% -- replaced with /var/log/ceph/cluster1-client.mirror.73940.log.gz 2026-02-19T20:44:31.345 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.73197.log 2026-02-19T20:44:31.346 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.74463.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.74463.log.gz 2026-02-19T20:44:31.346 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.68041.log 2026-02-19T20:44:31.346 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.73197.log: 92.4% -- replaced with /var/log/ceph/cluster2-client.mirror.73197.log.gz 2026-02-19T20:44:31.347 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.55765.log 2026-02-19T20:44:31.347 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.68041.log: 87.3% -- replaced with /var/log/ceph/cluster1-client.mirror.68041.log.gz 2026-02-19T20:44:31.347 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.76406.log 2026-02-19T20:44:31.350 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.55765.log: 93.3% -- replaced with /var/log/ceph/cluster2-client.mirror.55765.log.gz 2026-02-19T20:44:31.351 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.75083.log 2026-02-19T20:44:31.352 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.76406.log: 87.6% -- replaced with /var/log/ceph/cluster2-client.mirror.76406.log.gz 2026-02-19T20:44:31.352 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84642.log 2026-02-19T20:44:31.352 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.75083.log: 87.3% -- replaced with /var/log/ceph/cluster2-client.mirror.75083.log.gz 2026-02-19T20:44:31.353 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.58014.log 2026-02-19T20:44:31.354 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84642.log: 87.5% -- replaced with /var/log/ceph/cluster2-client.mirror.84642.log.gz 2026-02-19T20:44:31.354 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster1-client.mirror.77745.log 2026-02-19T20:44:31.355 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.58014.log: 88.6% -- replaced with /var/log/ceph/cluster2-client.mirror.58014.log.gz 2026-02-19T20:44:31.356 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.84091.log 2026-02-19T20:44:31.356 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.52680.log.gz 2026-02-19T20:44:31.356 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster1-client.mirror.77745.log: 86.4%gzip -5 --verbose -- /var/log/ceph/cluster2-client.mirror.65812.log 2026-02-19T20:44:31.356 INFO:teuthology.orchestra.run.trial080.stderr: -- replaced with /var/log/ceph/cluster1-client.mirror.77745.log.gz 2026-02-19T20:44:31.357 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.84091.log: 86.9% -- replaced with /var/log/ceph/cluster2-client.mirror.84091.log.gz 2026-02-19T20:44:31.357 INFO:teuthology.orchestra.run.trial080.stderr:/var/log/ceph/cluster2-client.mirror.65812.log: 88.4% -- replaced with /var/log/ceph/cluster2-client.mirror.65812.log.gz 2026-02-19T20:44:31.387 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.83350.log.gz 2026-02-19T20:44:31.425 INFO:teuthology.orchestra.run.trial080.stderr: 92.0% -- replaced with /var/log/ceph/cluster1-client.mirror.1.54729.log.gz 2026-02-19T20:44:31.454 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59221.log.gz 2026-02-19T20:44:31.462 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59709.log.gz 2026-02-19T20:44:31.477 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57819.log.gz 2026-02-19T20:44:31.505 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.82779.log.gz 2026-02-19T20:44:31.533 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.50619.log.gz 2026-02-19T20:44:31.554 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.58782.log.gz 2026-02-19T20:44:31.558 INFO:teuthology.orchestra.run.trial080.stderr: 93.6% -- replaced with /var/log/ceph/cluster2-client.mirror.61515.log.gz 2026-02-19T20:44:31.564 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.60896.log.gz 2026-02-19T20:44:31.573 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.60074.log.gz 2026-02-19T20:44:31.585 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.84491.log.gz 2026-02-19T20:44:31.593 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.53727.log.gz 2026-02-19T20:44:31.611 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59142.log.gz 2026-02-19T20:44:31.623 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.57229.log.gz 2026-02-19T20:44:31.630 INFO:teuthology.orchestra.run.trial080.stderr: 93.8% -- replaced with /var/log/ceph/cluster2-client.mirror.49874.log.gz 2026-02-19T20:44:31.662 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.61335.log.gz 2026-02-19T20:44:31.666 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.59063.log.gz 2026-02-19T20:44:31.722 INFO:teuthology.orchestra.run.trial080.stderr: 93.7% -- replaced with /var/log/ceph/cluster2-client.mirror.60474.log.gz 2026-02-19T20:44:34.150 INFO:teuthology.orchestra.run.trial073.stderr: 94.3% -- replaced with /var/log/ceph/cluster1-osd.0.log.gz 2026-02-19T20:44:34.183 INFO:teuthology.orchestra.run.trial073.stderr: 94.4% -- replaced with /var/log/ceph/cluster1-osd.1.log.gz 2026-02-19T20:44:34.631 INFO:teuthology.orchestra.run.trial073.stderr: 94.5% -- replaced with /var/log/ceph/cluster1-osd.2.log.gz 2026-02-19T20:44:34.632 INFO:teuthology.orchestra.run.trial073.stderr: 2026-02-19T20:44:34.632 INFO:teuthology.orchestra.run.trial073.stderr:real 0m5.100s 2026-02-19T20:44:34.632 INFO:teuthology.orchestra.run.trial073.stderr:user 0m14.414s 2026-02-19T20:44:34.632 INFO:teuthology.orchestra.run.trial073.stderr:sys 0m0.741s 2026-02-19T20:45:47.349 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-osd.2.log.gz 2026-02-19T20:45:49.976 INFO:teuthology.orchestra.run.trial080.stderr: 93.3% -- replaced with /var/log/ceph/cluster2-osd.1.log.gz 2026-02-19T20:46:16.941 INFO:teuthology.orchestra.run.trial080.stderr: 93.4% -- replaced with /var/log/ceph/cluster2-osd.0.log.gz 2026-02-19T20:46:16.942 INFO:teuthology.orchestra.run.trial080.stderr: 2026-02-19T20:46:16.942 INFO:teuthology.orchestra.run.trial080.stderr:real 1m47.381s 2026-02-19T20:46:16.942 INFO:teuthology.orchestra.run.trial080.stderr:user 4m49.999s 2026-02-19T20:46:16.942 INFO:teuthology.orchestra.run.trial080.stderr:sys 0m14.212s 2026-02-19T20:46:16.943 INFO:tasks.ceph:Archiving logs... 2026-02-19T20:46:16.943 DEBUG:teuthology.misc:Transferring archived files from trial073:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606/remote/trial073/log 2026-02-19T20:46:16.944 DEBUG:teuthology.orchestra.run.trial073:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-19T20:46:18.089 DEBUG:teuthology.misc:Transferring archived files from trial080:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606/remote/trial080/log 2026-02-19T20:46:18.090 DEBUG:teuthology.orchestra.run.trial080:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-19T20:46:40.476 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-19T20:46:40.490 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-19T20:46:40.491 DEBUG:teuthology.orchestra.run.trial073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-19T20:46:40.493 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-19T20:46:40.646 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-19T20:46:40.657 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-19T20:46:40.657 DEBUG:teuthology.orchestra.run.trial073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T20:46:40.659 DEBUG:teuthology.orchestra.run.trial080:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T20:46:40.674 INFO:teuthology.orchestra.run.trial073.stderr:bash: line 1: ntpq: command not found 2026-02-19T20:46:40.676 INFO:teuthology.orchestra.run.trial080.stderr:bash: line 1: ntpq: command not found 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial073.stdout:=============================================================================== 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial073.stdout:^* pool-96-231-54-150.washd> 1 8 377 66 +982us[+1056us] +/- 6894us 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial073.stdout:^+ chi3.us.ntp.li 2 6 377 1 -835us[ -835us] +/- 11ms 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial073.stdout:^- 23.186.168.129 2 7 375 61 -434us[ -434us] +/- 55ms 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial073.stdout:^- 66.42.71.197.vultruserco> 2 7 377 131 -642us[ -572us] +/- 47ms 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial080.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial080.stdout:=============================================================================== 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial080.stdout:^* pool-96-231-54-150.washd> 1 7 377 60 +870us[ +900us] +/- 6411us 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial080.stdout:^+ chi3.us.ntp.li 2 6 377 63 -628us[ -599us] +/- 11ms 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial080.stdout:^- 23.186.168.129 2 7 375 61 -232us[ -202us] +/- 55ms 2026-02-19T20:46:40.704 INFO:teuthology.orchestra.run.trial080.stdout:^- 66.42.71.197.vultruserco> 2 6 377 129 -462us[ -441us] +/- 47ms 2026-02-19T20:46:40.705 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-19T20:46:40.716 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-19T20:46:40.716 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-19T20:46:40.731 DEBUG:teuthology.orchestra.run.trial073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-19T20:46:40.800 DEBUG:teuthology.orchestra.run.trial080:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-19T20:46:40.858 DEBUG:teuthology.orchestra.run.trial073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-19T20:46:40.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T20:46:40.886 DEBUG:teuthology.orchestra.run.trial080:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-19T20:46:40.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T20:46:40.914 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-19T20:46:40.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-19T20:46:40.955 INFO:teuthology.task.internal:Duration was 2628.747674 seconds 2026-02-19T20:46:40.956 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-19T20:46:40.977 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-19T20:46:40.977 DEBUG:teuthology.orchestra.run.trial073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-19T20:46:40.980 DEBUG:teuthology.orchestra.run.trial080:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-19T20:46:41.016 INFO:teuthology.orchestra.run.trial073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-19T20:46:41.018 INFO:teuthology.orchestra.run.trial080.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-19T20:46:41.346 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-19T20:46:41.347 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial073.front.sepia.ceph.com 2026-02-19T20:46:41.347 DEBUG:teuthology.orchestra.run.trial073:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-19T20:46:41.368 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial080.front.sepia.ceph.com 2026-02-19T20:46:41.369 DEBUG:teuthology.orchestra.run.trial080:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-19T20:46:41.406 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-19T20:46:41.406 DEBUG:teuthology.orchestra.run.trial073:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T20:46:41.410 DEBUG:teuthology.orchestra.run.trial080:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T20:46:41.509 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-19T20:46:41.509 DEBUG:teuthology.orchestra.run.trial073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-19T20:46:41.511 DEBUG:teuthology.orchestra.run.trial080:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-19T20:46:41.532 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T20:46:41.532 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T20:46:41.533 INFO:teuthology.orchestra.run.trial073.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-19T20:46:41.533 INFO:teuthology.orchestra.run.trial073.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T20:46:41.533 INFO:teuthology.orchestra.run.trial073.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-19T20:46:41.534 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T20:46:41.535 INFO:teuthology.orchestra.run.trial080.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T20:46:41.535 INFO:teuthology.orchestra.run.trial080.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T20:46:41.535 INFO:teuthology.orchestra.run.trial080.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-19T20:46:41.536 INFO:teuthology.orchestra.run.trial080.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-19T20:46:41.542 INFO:teuthology.orchestra.run.trial073.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-19T20:46:41.545 INFO:teuthology.orchestra.run.trial080.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-19T20:46:41.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-19T20:46:41.558 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-19T20:46:41.558 DEBUG:teuthology.orchestra.run.trial073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-19T20:46:41.610 DEBUG:teuthology.orchestra.run.trial080:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-19T20:46:41.635 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-19T20:46:41.658 DEBUG:teuthology.orchestra.run.trial073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-19T20:46:41.660 DEBUG:teuthology.orchestra.run.trial080:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-19T20:46:41.681 INFO:teuthology.orchestra.run.trial073.stdout:kernel.core_pattern = core 2026-02-19T20:46:41.698 INFO:teuthology.orchestra.run.trial080.stdout:kernel.core_pattern = core 2026-02-19T20:46:41.713 DEBUG:teuthology.orchestra.run.trial073:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-19T20:46:41.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T20:46:41.751 DEBUG:teuthology.orchestra.run.trial080:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-19T20:46:41.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T20:46:41.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-19T20:46:41.791 INFO:teuthology.task.internal:Transferring archived files... 2026-02-19T20:46:41.792 DEBUG:teuthology.misc:Transferring archived files from trial073:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606/remote/trial073 2026-02-19T20:46:41.793 DEBUG:teuthology.orchestra.run.trial073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-19T20:46:41.830 DEBUG:teuthology.misc:Transferring archived files from trial080:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-19_15:47:56-rbd-wip-yuri-testing-2026-02-18-1820-distro-default-trial/59606/remote/trial080 2026-02-19T20:46:41.830 DEBUG:teuthology.orchestra.run.trial080:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-19T20:46:41.864 INFO:teuthology.task.internal:Removing archive directory... 2026-02-19T20:46:41.865 DEBUG:teuthology.orchestra.run.trial073:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-19T20:46:41.867 DEBUG:teuthology.orchestra.run.trial080:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-19T20:46:41.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-19T20:46:41.931 INFO:teuthology.task.internal:Not uploading archives. 2026-02-19T20:46:41.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-19T20:46:41.945 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-19T20:46:41.945 DEBUG:teuthology.orchestra.run.trial073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-19T20:46:41.948 DEBUG:teuthology.orchestra.run.trial080:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-19T20:46:41.963 INFO:teuthology.orchestra.run.trial073.stdout: 917572 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 19 20:46 /home/ubuntu/cephtest 2026-02-19T20:46:41.971 INFO:teuthology.orchestra.run.trial080.stdout: 917572 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 19 20:46 /home/ubuntu/cephtest 2026-02-19T20:46:41.972 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-19T20:46:41.989 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-19T20:46:42.025 INFO:teuthology.run:Summary data: description: rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zlib policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{centos_latest} workloads/rbd-mirror-snapshot-stress-workunit-minimum} duration: 2628.7476744651794 flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com success: true 2026-02-19T20:46:42.025 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-19T20:46:42.126 INFO:teuthology.run:pass